2026-02-20T17:52:24.750 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-20T17:52:24.765 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-20T17:52:24.841 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/yuriw-2026-02-20_16:06:38-rados-wip-pdonnell-testing-20260219.182737-tentacle-distro-default-trial/60966 branch: wip-pdonnell-testing-20260219.182737-tentacle description: rados/standalone/{supported-random-distro$/{centos_latest} workloads/erasure-code} email: yweinste@redhat.com first_in_suite: false flavor: default job_id: '60966' kernel: branch: distro kdb: true sha1: distro last_in_suite: false machine_type: trial name: yuriw-2026-02-20_16:06:38-rados-wip-pdonnell-testing-20260219.182737-tentacle-distro-default-trial no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20260219.182737-tentacle ceph: conf: mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: f7870454149f22574086dd2ff55d050585136531 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: f7870454149f22574086dd2ff55d050585136531 workunit: branch: wip-pdonnell-testing-20260219.182737-tentacle sha1: f7870454149f22574086dd2ff55d050585136531 owner: scheduled_yuriw@soko04.front.sepia.ceph.com priority: 69 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 8419 sha1: f7870454149f22574086dd2ff55d050585136531 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-pdonnell-testing-20260219.182737-tentacle suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_f7870454149f22574086dd2ff55d050585136531/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: f7870454149f22574086dd2ff55d050585136531 targets: trial195.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJyNILvltbiP0R5a6gYy2FInfREnDrQOYUZNbni0O3EkKUtPBU9lPIxrla6RlafVHXq6X+0oTDB8drNdipTfTzo= tasks: - install: null - workunit: basedir: qa/standalone clients: all: - erasure-code teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-20_16:06:38 tube: trial user: yuriw verbose: true worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.1540699 2026-02-20T17:52:24.841 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_f7870454149f22574086dd2ff55d050585136531/qa; will attempt to use it 2026-02-20T17:52:24.842 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_f7870454149f22574086dd2ff55d050585136531/qa/tasks 2026-02-20T17:52:24.842 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-20T17:52:24.844 INFO:teuthology.task.internal:Checking packages... 2026-02-20T17:52:24.844 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'f7870454149f22574086dd2ff55d050585136531' 2026-02-20T17:52:24.844 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-20T17:52:24.844 INFO:teuthology.packaging:ref: None 2026-02-20T17:52:24.844 INFO:teuthology.packaging:tag: None 2026-02-20T17:52:24.844 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20260219.182737-tentacle 2026-02-20T17:52:24.844 INFO:teuthology.packaging:sha1: f7870454149f22574086dd2ff55d050585136531 2026-02-20T17:52:24.844 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20260219.182737-tentacle 2026-02-20T17:52:24.974 INFO:teuthology.task.internal:Found packages for ceph version 20.2.0-677.gf7870454 2026-02-20T17:52:24.975 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-20T17:52:24.987 INFO:teuthology.task.internal:no buildpackages task found 2026-02-20T17:52:24.987 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-20T17:52:25.001 INFO:teuthology.task.internal:Saving configuration 2026-02-20T17:52:25.009 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-20T17:52:25.022 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-20T17:52:25.104 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial195.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-20_16:06:38-rados-wip-pdonnell-testing-20260219.182737-tentacle-distro-default-trial/60966', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-20 17:42:19.748174', 'locked_by': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJyNILvltbiP0R5a6gYy2FInfREnDrQOYUZNbni0O3EkKUtPBU9lPIxrla6RlafVHXq6X+0oTDB8drNdipTfTzo='} 2026-02-20T17:52:25.105 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-20T17:52:25.116 INFO:teuthology.task.internal:roles: ubuntu@trial195.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-20T17:52:25.116 INFO:teuthology.run_tasks:Running task console_log... 2026-02-20T17:52:25.191 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe4eea1a700>, signals=[15]) 2026-02-20T17:52:25.191 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-20T17:52:25.205 INFO:teuthology.task.internal:Opening connections... 2026-02-20T17:52:25.205 DEBUG:teuthology.task.internal:connecting to ubuntu@trial195.front.sepia.ceph.com 2026-02-20T17:52:25.206 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-20T17:52:25.268 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-20T17:52:25.280 DEBUG:teuthology.orchestra.run.trial195:> uname -m 2026-02-20T17:52:25.347 INFO:teuthology.orchestra.run.trial195.stdout:x86_64 2026-02-20T17:52:25.348 DEBUG:teuthology.orchestra.run.trial195:> cat /etc/os-release 2026-02-20T17:52:25.404 INFO:teuthology.orchestra.run.trial195.stdout:NAME="CentOS Stream" 2026-02-20T17:52:25.405 INFO:teuthology.orchestra.run.trial195.stdout:VERSION="9" 2026-02-20T17:52:25.405 INFO:teuthology.orchestra.run.trial195.stdout:ID="centos" 2026-02-20T17:52:25.405 INFO:teuthology.orchestra.run.trial195.stdout:ID_LIKE="rhel fedora" 2026-02-20T17:52:25.405 INFO:teuthology.orchestra.run.trial195.stdout:VERSION_ID="9" 2026-02-20T17:52:25.405 INFO:teuthology.orchestra.run.trial195.stdout:PLATFORM_ID="platform:el9" 2026-02-20T17:52:25.405 INFO:teuthology.orchestra.run.trial195.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-20T17:52:25.405 INFO:teuthology.orchestra.run.trial195.stdout:ANSI_COLOR="0;31" 2026-02-20T17:52:25.405 INFO:teuthology.orchestra.run.trial195.stdout:LOGO="fedora-logo-icon" 2026-02-20T17:52:25.405 INFO:teuthology.orchestra.run.trial195.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-20T17:52:25.405 INFO:teuthology.orchestra.run.trial195.stdout:HOME_URL="https://centos.org/" 2026-02-20T17:52:25.405 INFO:teuthology.orchestra.run.trial195.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-20T17:52:25.405 INFO:teuthology.orchestra.run.trial195.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-20T17:52:25.405 INFO:teuthology.orchestra.run.trial195.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-20T17:52:25.406 INFO:teuthology.lock.ops:Updating trial195.front.sepia.ceph.com on lock server 2026-02-20T17:52:25.493 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-20T17:52:25.506 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-20T17:52:25.518 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-20T17:52:25.518 DEBUG:teuthology.orchestra.run.trial195:> test '!' -e /home/ubuntu/cephtest 2026-02-20T17:52:25.535 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-20T17:52:25.547 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-20T17:52:25.547 DEBUG:teuthology.orchestra.run.trial195:> test -z $(ls -A /var/lib/ceph) 2026-02-20T17:52:25.592 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-20T17:52:25.696 INFO:teuthology.run_tasks:Running task kernel... 2026-02-20T17:52:25.717 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': True, 'sha1': 'distro'} 2026-02-20T17:52:25.717 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, timeout 300 2026-02-20T17:52:25.718 DEBUG:teuthology.orchestra.run.trial195:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-20T17:52:25.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-20T17:52:25.734 DEBUG:teuthology.orchestra.run.trial195:> uname -r 2026-02-20T17:52:25.790 INFO:teuthology.orchestra.run.trial195.stdout:5.14.0-681.el9.x86_64 2026-02-20T17:52:25.790 INFO:teuthology.task.kernel:Running kernel on trial195: 5.14.0-681.el9.x86_64 2026-02-20T17:52:25.790 DEBUG:teuthology.orchestra.run.trial195:> sudo yum install -y kernel 2026-02-20T17:52:34.819 INFO:teuthology.orchestra.run.trial195.stdout:CentOS Stream 9 - BaseOS 1.0 MB/s | 8.9 MB 00:08 2026-02-20T17:53:01.663 INFO:teuthology.orchestra.run.trial195.stdout:CentOS Stream 9 - AppStream 1.0 MB/s | 27 MB 00:26 2026-02-20T17:53:04.423 INFO:teuthology.orchestra.run.trial195.stdout:CentOS Stream 9 - CRB 21 MB/s | 7.8 MB 00:00 2026-02-20T17:53:06.259 INFO:teuthology.orchestra.run.trial195.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 20 kB 00:01 2026-02-20T17:53:06.819 INFO:teuthology.orchestra.run.trial195.stdout:Extra Packages for Enterprise Linux 41 MB/s | 20 MB 00:00 2026-02-20T17:53:09.987 INFO:teuthology.orchestra.run.trial195.stdout:lab-extras 1.8 MB/s | 50 kB 00:00 2026-02-20T17:53:10.997 INFO:teuthology.orchestra.run.trial195.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-20T17:53:10.997 INFO:teuthology.orchestra.run.trial195.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-20T17:53:11.016 INFO:teuthology.orchestra.run.trial195.stdout:Dependencies resolved. 2026-02-20T17:53:11.017 INFO:teuthology.orchestra.run.trial195.stdout:Nothing to do. 2026-02-20T17:53:11.017 INFO:teuthology.orchestra.run.trial195.stdout:Complete! 2026-02-20T17:53:11.041 DEBUG:teuthology.orchestra.run.trial195:> echo no | sudo yum reinstall kernel || true 2026-02-20T17:53:11.339 INFO:teuthology.orchestra.run.trial195.stdout:Last metadata expiration check: 0:00:02 ago on Fri 20 Feb 2026 05:53:09 PM UTC. 2026-02-20T17:53:11.421 INFO:teuthology.orchestra.run.trial195.stdout:Dependencies resolved. 2026-02-20T17:53:11.422 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:53:11.422 INFO:teuthology.orchestra.run.trial195.stdout: Package Architecture Version Repository Size 2026-02-20T17:53:11.422 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:53:11.423 INFO:teuthology.orchestra.run.trial195.stdout:Reinstalling: 2026-02-20T17:53:11.423 INFO:teuthology.orchestra.run.trial195.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-20T17:53:11.423 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:53:11.423 INFO:teuthology.orchestra.run.trial195.stdout:Transaction Summary 2026-02-20T17:53:11.423 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:53:11.423 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:53:11.423 INFO:teuthology.orchestra.run.trial195.stdout:Total download size: 803 k 2026-02-20T17:53:11.423 INFO:teuthology.orchestra.run.trial195.stdout:Installed size: 0 2026-02-20T17:53:11.423 INFO:teuthology.orchestra.run.trial195.stderr:Operation aborted. 2026-02-20T17:53:11.442 INFO:teuthology.orchestra.run.trial195.stdout:Is this ok [y/N]: 2026-02-20T17:53:11.443 DEBUG:teuthology.orchestra.run.trial195:> sudo yum reinstall -y kernel || true 2026-02-20T17:53:11.746 INFO:teuthology.orchestra.run.trial195.stdout:Last metadata expiration check: 0:00:02 ago on Fri 20 Feb 2026 05:53:09 PM UTC. 2026-02-20T17:53:11.828 INFO:teuthology.orchestra.run.trial195.stdout:Dependencies resolved. 2026-02-20T17:53:11.828 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:53:11.828 INFO:teuthology.orchestra.run.trial195.stdout: Package Architecture Version Repository Size 2026-02-20T17:53:11.828 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:53:11.829 INFO:teuthology.orchestra.run.trial195.stdout:Reinstalling: 2026-02-20T17:53:11.829 INFO:teuthology.orchestra.run.trial195.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-20T17:53:11.829 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:53:11.829 INFO:teuthology.orchestra.run.trial195.stdout:Transaction Summary 2026-02-20T17:53:11.829 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:53:11.829 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:53:11.829 INFO:teuthology.orchestra.run.trial195.stdout:Total download size: 803 k 2026-02-20T17:53:11.829 INFO:teuthology.orchestra.run.trial195.stdout:Installed size: 0 2026-02-20T17:53:11.829 INFO:teuthology.orchestra.run.trial195.stdout:Downloading Packages: 2026-02-20T17:53:13.529 INFO:teuthology.orchestra.run.trial195.stdout:kernel-5.14.0-681.el9.x86_64.rpm 565 kB/s | 803 kB 00:01 2026-02-20T17:53:13.530 INFO:teuthology.orchestra.run.trial195.stdout:-------------------------------------------------------------------------------- 2026-02-20T17:53:13.530 INFO:teuthology.orchestra.run.trial195.stdout:Total 472 kB/s | 803 kB 00:01 2026-02-20T17:53:13.542 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction check 2026-02-20T17:53:13.575 INFO:teuthology.orchestra.run.trial195.stdout:Transaction check succeeded. 2026-02-20T17:53:13.575 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction test 2026-02-20T17:53:13.580 INFO:teuthology.orchestra.run.trial195.stdout:Transaction test succeeded. 2026-02-20T17:53:13.580 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction 2026-02-20T17:53:13.604 INFO:teuthology.orchestra.run.trial195.stdout: Preparing : 1/1 2026-02-20T17:53:13.607 INFO:teuthology.orchestra.run.trial195.stdout: Reinstalling : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-20T17:53:13.607 INFO:teuthology.orchestra.run.trial195.stdout: Cleanup : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-20T17:53:13.713 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: kernel-5.14.0-681.el9.x86_64 2/2 2026-02-20T17:53:13.713 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-20T17:53:13.768 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-20T17:53:13.768 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:53:13.768 INFO:teuthology.orchestra.run.trial195.stdout:Reinstalled: 2026-02-20T17:53:13.769 INFO:teuthology.orchestra.run.trial195.stdout: kernel-5.14.0-681.el9.x86_64 2026-02-20T17:53:13.769 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:53:13.769 INFO:teuthology.orchestra.run.trial195.stdout:Complete! 2026-02-20T17:53:13.795 DEBUG:teuthology.orchestra.run.trial195:> rpm -q kernel | sort -rV | head -n 1 2026-02-20T17:53:13.862 INFO:teuthology.orchestra.run.trial195.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-20T17:53:13.862 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-20T17:53:13.862 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2026-02-20T17:53:13.862 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-20T17:53:13.863 DEBUG:teuthology.orchestra.run.trial195:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-20T17:53:13.926 INFO:teuthology.orchestra.run.trial195.stdout:ttyS1 2026-02-20T17:53:13.946 DEBUG:teuthology.parallel:result is None 2026-02-20T17:53:13.946 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-20T17:53:13.958 INFO:teuthology.task.internal:Creating test directory... 2026-02-20T17:53:13.959 DEBUG:teuthology.orchestra.run.trial195:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-20T17:53:14.003 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-20T17:53:14.015 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-20T17:53:14.028 INFO:teuthology.task.internal:Creating archive directory... 2026-02-20T17:53:14.029 DEBUG:teuthology.orchestra.run.trial195:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-20T17:53:14.065 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-20T17:53:14.079 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-20T17:53:14.079 DEBUG:teuthology.orchestra.run.trial195:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-20T17:53:14.125 INFO:teuthology.orchestra.run.trial195.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:53:14.134 INFO:teuthology.orchestra.run.trial195.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:53:14.136 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-20T17:53:14.148 INFO:teuthology.task.internal:Configuring sudo... 2026-02-20T17:53:14.148 DEBUG:teuthology.orchestra.run.trial195:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-20T17:53:14.202 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-20T17:53:14.218 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-20T17:53:14.218 DEBUG:teuthology.orchestra.run.trial195:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-20T17:53:14.259 DEBUG:teuthology.orchestra.run.trial195:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-20T17:53:14.420 DEBUG:teuthology.orchestra.run.trial195:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-20T17:53:14.445 DEBUG:teuthology.orchestra.run.trial195:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-20T17:53:14.595 DEBUG:teuthology.orchestra.run.trial195:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-20T17:53:14.620 DEBUG:teuthology.orchestra.run.trial195:> set -ex 2026-02-20T17:53:14.620 DEBUG:teuthology.orchestra.run.trial195:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-20T17:53:14.685 DEBUG:teuthology.orchestra.run.trial195:> sudo service rsyslog restart 2026-02-20T17:53:14.756 INFO:teuthology.orchestra.run.trial195.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-20T17:53:15.261 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-20T17:53:15.272 INFO:teuthology.task.internal:Starting timer... 2026-02-20T17:53:15.272 INFO:teuthology.run_tasks:Running task pcp... 2026-02-20T17:53:15.291 INFO:teuthology.run_tasks:Running task selinux... 2026-02-20T17:53:15.309 DEBUG:teuthology.orchestra.run.trial195:> sudo service auditd rotate 2026-02-20T17:53:15.336 INFO:teuthology.orchestra.run.trial195.stdout:Rotating logs: 2026-02-20T17:53:15.337 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-20T17:53:15.338 DEBUG:teuthology.orchestra.run.trial195:> /usr/sbin/getenforce 2026-02-20T17:53:15.394 INFO:teuthology.orchestra.run.trial195.stdout:Permissive 2026-02-20T17:53:15.394 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial195.front.sepia.ceph.com': 'permissive'} 2026-02-20T17:53:15.395 DEBUG:teuthology.orchestra.run.trial195:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-20T17:53:15.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-20T17:53:15.459 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-20T17:53:15.459 DEBUG:teuthology.orchestra.run.trial195:> sudo /usr/sbin/setenforce permissive 2026-02-20T17:53:15.525 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-20T17:53:15.555 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2026-02-20T17:53:15.559 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2026-02-20T17:53:15.690 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-20T17:53:15.696 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-20T17:53:15.697 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial195.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-20T17:55:58.945 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial195.front.sepia.ceph.com')] 2026-02-20T17:55:58.946 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial195.front.sepia.ceph.com' 2026-02-20T17:55:58.947 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-20T17:55:59.056 DEBUG:teuthology.orchestra.run.trial195:> true 2026-02-20T17:55:59.129 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial195.front.sepia.ceph.com' 2026-02-20T17:55:59.129 INFO:teuthology.run_tasks:Running task clock... 2026-02-20T17:55:59.146 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-20T17:55:59.147 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-20T17:55:59.147 DEBUG:teuthology.orchestra.run.trial195:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-20T17:55:59.195 INFO:teuthology.orchestra.run.trial195.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-20T17:55:59.208 INFO:teuthology.orchestra.run.trial195.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-20T17:55:59.228 INFO:teuthology.orchestra.run.trial195.stderr:sudo: ntpd: command not found 2026-02-20T17:55:59.238 INFO:teuthology.orchestra.run.trial195.stdout:506 Cannot talk to daemon 2026-02-20T17:55:59.251 INFO:teuthology.orchestra.run.trial195.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-20T17:55:59.263 INFO:teuthology.orchestra.run.trial195.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-20T17:55:59.314 INFO:teuthology.orchestra.run.trial195.stderr:bash: line 1: ntpq: command not found 2026-02-20T17:55:59.316 INFO:teuthology.orchestra.run.trial195.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-20T17:55:59.316 INFO:teuthology.orchestra.run.trial195.stdout:=============================================================================== 2026-02-20T17:55:59.317 INFO:teuthology.run_tasks:Running task install... 2026-02-20T17:55:59.331 DEBUG:teuthology.task.install:project ceph 2026-02-20T17:55:59.331 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'f7870454149f22574086dd2ff55d050585136531'}} 2026-02-20T17:55:59.331 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'f7870454149f22574086dd2ff55d050585136531'} 2026-02-20T17:55:59.331 INFO:teuthology.task.install:Using flavor: default 2026-02-20T17:55:59.340 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-20T17:55:59.341 INFO:teuthology.task.install:extra packages: [] 2026-02-20T17:55:59.341 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f7870454149f22574086dd2ff55d050585136531', 'tag': None, 'wait_for_package': False} 2026-02-20T17:55:59.342 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f7870454149f22574086dd2ff55d050585136531 2026-02-20T17:55:59.510 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20260219.182737-tentacle/f7870454149f22574086dd2ff55d050585136531/centos/9/flavors/default/ 2026-02-20T17:55:59.510 INFO:teuthology.task.install.rpm:Package version is 20.2.0-677.gf7870454 2026-02-20T17:55:59.649 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20260219.182737-tentacle/f7870454149f22574086dd2ff55d050585136531/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20260219.182737-tentacle/f7870454149f22574086dd2ff55d050585136531/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20260219.182737-tentacle/f7870454149f22574086dd2ff55d050585136531/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-20T17:55:59.649 DEBUG:teuthology.orchestra.run.trial195:> set -ex 2026-02-20T17:55:59.649 DEBUG:teuthology.orchestra.run.trial195:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-20T17:55:59.675 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-20T17:55:59.676 DEBUG:teuthology.orchestra.run.trial195:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f7870454149f22574086dd2ff55d050585136531/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-20T17:55:59.740 DEBUG:teuthology.orchestra.run.trial195:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-20T17:55:59.815 DEBUG:teuthology.orchestra.run.trial195:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-20T17:55:59.879 INFO:teuthology.orchestra.run.trial195.stdout:check_obsoletes = 1 2026-02-20T17:55:59.880 DEBUG:teuthology.orchestra.run.trial195:> sudo yum clean all 2026-02-20T17:56:00.039 INFO:teuthology.orchestra.run.trial195.stdout:41 files removed 2026-02-20T17:56:00.053 DEBUG:teuthology.orchestra.run.trial195:> sudo yum -y install ceph-radosgw 2026-02-20T17:56:00.324 INFO:teuthology.orchestra.run.trial195.stdout:ceph packages for x86_64 679 kB/s | 87 kB 00:00 2026-02-20T17:56:00.464 INFO:teuthology.orchestra.run.trial195.stdout:ceph noarch packages 158 kB/s | 18 kB 00:00 2026-02-20T17:56:00.589 INFO:teuthology.orchestra.run.trial195.stdout:ceph source packages 19 kB/s | 1.9 kB 00:00 2026-02-20T17:56:01.650 INFO:teuthology.orchestra.run.trial195.stdout:CentOS Stream 9 - BaseOS 8.6 MB/s | 8.9 MB 00:01 2026-02-20T17:56:04.783 INFO:teuthology.orchestra.run.trial195.stdout:CentOS Stream 9 - AppStream 10 MB/s | 27 MB 00:02 2026-02-20T17:56:08.233 INFO:teuthology.orchestra.run.trial195.stdout:CentOS Stream 9 - CRB 7.2 MB/s | 7.8 MB 00:01 2026-02-20T17:56:09.878 INFO:teuthology.orchestra.run.trial195.stdout:CentOS Stream 9 - Extras packages 22 kB/s | 20 kB 00:00 2026-02-20T17:56:10.980 INFO:teuthology.orchestra.run.trial195.stdout:Extra Packages for Enterprise Linux 20 MB/s | 20 MB 00:01 2026-02-20T17:56:14.178 INFO:teuthology.orchestra.run.trial195.stdout:lab-extras 1.8 MB/s | 50 kB 00:00 2026-02-20T17:56:15.366 INFO:teuthology.orchestra.run.trial195.stdout:Dependencies resolved. 2026-02-20T17:56:15.368 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:56:15.368 INFO:teuthology.orchestra.run.trial195.stdout: Package Arch Version Repository Size 2026-02-20T17:56:15.368 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:56:15.368 INFO:teuthology.orchestra.run.trial195.stdout:Installing: 2026-02-20T17:56:15.368 INFO:teuthology.orchestra.run.trial195.stdout: ceph-radosgw x86_64 2:20.2.0-677.gf7870454.el9 ceph 24 M 2026-02-20T17:56:15.368 INFO:teuthology.orchestra.run.trial195.stdout:Upgrading: 2026-02-20T17:56:15.368 INFO:teuthology.orchestra.run.trial195.stdout: librados2 x86_64 2:20.2.0-677.gf7870454.el9 ceph 3.5 M 2026-02-20T17:56:15.368 INFO:teuthology.orchestra.run.trial195.stdout: librbd1 x86_64 2:20.2.0-677.gf7870454.el9 ceph 2.8 M 2026-02-20T17:56:15.369 INFO:teuthology.orchestra.run.trial195.stdout:Installing dependencies: 2026-02-20T17:56:15.369 INFO:teuthology.orchestra.run.trial195.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-20T17:56:15.369 INFO:teuthology.orchestra.run.trial195.stdout: ceph-base x86_64 2:20.2.0-677.gf7870454.el9 ceph 5.9 M 2026-02-20T17:56:15.369 INFO:teuthology.orchestra.run.trial195.stdout: ceph-common x86_64 2:20.2.0-677.gf7870454.el9 ceph 24 M 2026-02-20T17:56:15.369 INFO:teuthology.orchestra.run.trial195.stdout: ceph-selinux x86_64 2:20.2.0-677.gf7870454.el9 ceph 25 k 2026-02-20T17:56:15.369 INFO:teuthology.orchestra.run.trial195.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-20T17:56:15.369 INFO:teuthology.orchestra.run.trial195.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-20T17:56:15.369 INFO:teuthology.orchestra.run.trial195.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-20T17:56:15.369 INFO:teuthology.orchestra.run.trial195.stdout: libcephfs2 x86_64 2:20.2.0-677.gf7870454.el9 ceph 869 k 2026-02-20T17:56:15.369 INFO:teuthology.orchestra.run.trial195.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-20T17:56:15.369 INFO:teuthology.orchestra.run.trial195.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-20T17:56:15.370 INFO:teuthology.orchestra.run.trial195.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-20T17:56:15.370 INFO:teuthology.orchestra.run.trial195.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-20T17:56:15.370 INFO:teuthology.orchestra.run.trial195.stdout: libradosstriper1 x86_64 2:20.2.0-677.gf7870454.el9 ceph 250 k 2026-02-20T17:56:15.370 INFO:teuthology.orchestra.run.trial195.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-20T17:56:15.370 INFO:teuthology.orchestra.run.trial195.stdout: librgw2 x86_64 2:20.2.0-677.gf7870454.el9 ceph 6.4 M 2026-02-20T17:56:15.370 INFO:teuthology.orchestra.run.trial195.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-20T17:56:15.370 INFO:teuthology.orchestra.run.trial195.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-20T17:56:15.370 INFO:teuthology.orchestra.run.trial195.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-20T17:56:15.370 INFO:teuthology.orchestra.run.trial195.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-20T17:56:15.370 INFO:teuthology.orchestra.run.trial195.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-20T17:56:15.370 INFO:teuthology.orchestra.run.trial195.stdout: python3-ceph-argparse x86_64 2:20.2.0-677.gf7870454.el9 ceph 45 k 2026-02-20T17:56:15.371 INFO:teuthology.orchestra.run.trial195.stdout: python3-ceph-common x86_64 2:20.2.0-677.gf7870454.el9 ceph 175 k 2026-02-20T17:56:15.371 INFO:teuthology.orchestra.run.trial195.stdout: python3-cephfs x86_64 2:20.2.0-677.gf7870454.el9 ceph 163 k 2026-02-20T17:56:15.371 INFO:teuthology.orchestra.run.trial195.stdout: python3-rados x86_64 2:20.2.0-677.gf7870454.el9 ceph 323 k 2026-02-20T17:56:15.371 INFO:teuthology.orchestra.run.trial195.stdout: python3-rbd x86_64 2:20.2.0-677.gf7870454.el9 ceph 305 k 2026-02-20T17:56:15.371 INFO:teuthology.orchestra.run.trial195.stdout: python3-rgw x86_64 2:20.2.0-677.gf7870454.el9 ceph 99 k 2026-02-20T17:56:15.371 INFO:teuthology.orchestra.run.trial195.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-20T17:56:15.371 INFO:teuthology.orchestra.run.trial195.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-20T17:56:15.371 INFO:teuthology.orchestra.run.trial195.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-20T17:56:15.371 INFO:teuthology.orchestra.run.trial195.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-20T17:56:15.371 INFO:teuthology.orchestra.run.trial195.stdout:Installing weak dependencies: 2026-02-20T17:56:15.371 INFO:teuthology.orchestra.run.trial195.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-20T17:56:15.371 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:15.372 INFO:teuthology.orchestra.run.trial195.stdout:Transaction Summary 2026-02-20T17:56:15.372 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:56:15.372 INFO:teuthology.orchestra.run.trial195.stdout:Install 32 Packages 2026-02-20T17:56:15.372 INFO:teuthology.orchestra.run.trial195.stdout:Upgrade 2 Packages 2026-02-20T17:56:15.372 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:15.373 INFO:teuthology.orchestra.run.trial195.stdout:Total download size: 78 M 2026-02-20T17:56:15.373 INFO:teuthology.orchestra.run.trial195.stdout:Downloading Packages: 2026-02-20T17:56:16.229 INFO:teuthology.orchestra.run.trial195.stdout:(1/34): ceph-base-20.2.0-677.gf7870454.el9.x86_ 18 MB/s | 5.9 MB 00:00 2026-02-20T17:56:16.264 INFO:teuthology.orchestra.run.trial195.stdout:(2/34): ceph-selinux-20.2.0-677.gf7870454.el9.x 722 kB/s | 25 kB 00:00 2026-02-20T17:56:16.318 INFO:teuthology.orchestra.run.trial195.stdout:(3/34): libcephfs2-20.2.0-677.gf7870454.el9.x86 16 MB/s | 869 kB 00:00 2026-02-20T17:56:16.361 INFO:teuthology.orchestra.run.trial195.stdout:(4/34): libradosstriper1-20.2.0-677.gf7870454.e 5.7 MB/s | 250 kB 00:00 2026-02-20T17:56:16.696 INFO:teuthology.orchestra.run.trial195.stdout:(5/34): librgw2-20.2.0-677.gf7870454.el9.x86_64 19 MB/s | 6.4 MB 00:00 2026-02-20T17:56:16.733 INFO:teuthology.orchestra.run.trial195.stdout:(6/34): python3-ceph-argparse-20.2.0-677.gf7870 1.2 MB/s | 45 kB 00:00 2026-02-20T17:56:16.771 INFO:teuthology.orchestra.run.trial195.stdout:(7/34): python3-ceph-common-20.2.0-677.gf787045 4.4 MB/s | 175 kB 00:00 2026-02-20T17:56:16.808 INFO:teuthology.orchestra.run.trial195.stdout:(8/34): python3-cephfs-20.2.0-677.gf7870454.el9 4.3 MB/s | 163 kB 00:00 2026-02-20T17:56:16.849 INFO:teuthology.orchestra.run.trial195.stdout:(9/34): python3-rados-20.2.0-677.gf7870454.el9. 7.7 MB/s | 323 kB 00:00 2026-02-20T17:56:16.890 INFO:teuthology.orchestra.run.trial195.stdout:(10/34): python3-rbd-20.2.0-677.gf7870454.el9.x 7.3 MB/s | 305 kB 00:00 2026-02-20T17:56:16.928 INFO:teuthology.orchestra.run.trial195.stdout:(11/34): python3-rgw-20.2.0-677.gf7870454.el9.x 2.6 MB/s | 99 kB 00:00 2026-02-20T17:56:17.011 INFO:teuthology.orchestra.run.trial195.stdout:(12/34): ceph-common-20.2.0-677.gf7870454.el9.x 21 MB/s | 24 MB 00:01 2026-02-20T17:56:17.096 INFO:teuthology.orchestra.run.trial195.stdout:(13/34): ceph-radosgw-20.2.0-677.gf7870454.el9. 20 MB/s | 24 MB 00:01 2026-02-20T17:56:17.498 INFO:teuthology.orchestra.run.trial195.stdout:(14/34): mailcap-2.1.49-5.el9.noarch.rpm 58 kB/s | 33 kB 00:00 2026-02-20T17:56:17.737 INFO:teuthology.orchestra.run.trial195.stdout:(15/34): pciutils-3.7.0-7.el9.x86_64.rpm 128 kB/s | 93 kB 00:00 2026-02-20T17:56:17.833 INFO:teuthology.orchestra.run.trial195.stdout:(16/34): libnbd-1.20.3-4.el9.x86_64.rpm 490 kB/s | 164 kB 00:00 2026-02-20T17:56:17.848 INFO:teuthology.orchestra.run.trial195.stdout:(17/34): boost-program-options-1.75.0-13.el9.x8 138 kB/s | 104 kB 00:00 2026-02-20T17:56:17.932 INFO:teuthology.orchestra.run.trial195.stdout:(18/34): librabbitmq-0.11.0-7.el9.x86_64.rpm 456 kB/s | 45 kB 00:00 2026-02-20T17:56:17.993 INFO:teuthology.orchestra.run.trial195.stdout:(19/34): libpmemobj-1.12.1-1.el9.x86_64.rpm 624 kB/s | 160 kB 00:00 2026-02-20T17:56:18.329 INFO:teuthology.orchestra.run.trial195.stdout:(20/34): qatlib-25.08.0-2.el9.x86_64.rpm 713 kB/s | 240 kB 00:00 2026-02-20T17:56:18.346 INFO:teuthology.orchestra.run.trial195.stdout:(21/34): lttng-ust-2.12.0-6.el9.x86_64.rpm 706 kB/s | 292 kB 00:00 2026-02-20T17:56:18.429 INFO:teuthology.orchestra.run.trial195.stdout:(22/34): qatlib-service-25.08.0-2.el9.x86_64.rp 371 kB/s | 37 kB 00:00 2026-02-20T17:56:18.446 INFO:teuthology.orchestra.run.trial195.stdout:(23/34): qatzip-libs-1.3.1-1.el9.x86_64.rpm 665 kB/s | 66 kB 00:00 2026-02-20T17:56:18.590 INFO:teuthology.orchestra.run.trial195.stdout:(24/34): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.9 MB/s | 308 kB 00:00 2026-02-20T17:56:18.610 INFO:teuthology.orchestra.run.trial195.stdout:(25/34): libarrow-doc-9.0.0-15.el9.noarch.rpm 1.2 MB/s | 25 kB 00:00 2026-02-20T17:56:18.630 INFO:teuthology.orchestra.run.trial195.stdout:(26/34): liboath-2.6.12-1.el9.x86_64.rpm 2.4 MB/s | 49 kB 00:00 2026-02-20T17:56:18.650 INFO:teuthology.orchestra.run.trial195.stdout:(27/34): libunwind-1.6.2-1.el9.x86_64.rpm 3.3 MB/s | 67 kB 00:00 2026-02-20T17:56:18.689 INFO:teuthology.orchestra.run.trial195.stdout:(28/34): libarrow-9.0.0-15.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2026-02-20T17:56:18.692 INFO:teuthology.orchestra.run.trial195.stdout:(29/34): parquet-libs-9.0.0-15.el9.x86_64.rpm 20 MB/s | 838 kB 00:00 2026-02-20T17:56:18.710 INFO:teuthology.orchestra.run.trial195.stdout:(30/34): re2-20211101-20.el9.x86_64.rpm 9.0 MB/s | 191 kB 00:00 2026-02-20T17:56:18.734 INFO:teuthology.orchestra.run.trial195.stdout:(31/34): thrift-0.15.0-4.el9.x86_64.rpm 37 MB/s | 1.6 MB 00:00 2026-02-20T17:56:18.837 INFO:teuthology.orchestra.run.trial195.stdout:(32/34): librados2-20.2.0-677.gf7870454.el9.x86 28 MB/s | 3.5 MB 00:00 2026-02-20T17:56:18.871 INFO:teuthology.orchestra.run.trial195.stdout:(33/34): librbd1-20.2.0-677.gf7870454.el9.x86_6 21 MB/s | 2.8 MB 00:00 2026-02-20T17:56:19.535 INFO:teuthology.orchestra.run.trial195.stdout:(34/34): librdkafka-1.6.1-102.el9.x86_64.rpm 393 kB/s | 662 kB 00:01 2026-02-20T17:56:19.542 INFO:teuthology.orchestra.run.trial195.stdout:-------------------------------------------------------------------------------- 2026-02-20T17:56:19.542 INFO:teuthology.orchestra.run.trial195.stdout:Total 19 MB/s | 78 MB 00:04 2026-02-20T17:56:19.633 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction check 2026-02-20T17:56:19.650 INFO:teuthology.orchestra.run.trial195.stdout:Transaction check succeeded. 2026-02-20T17:56:19.650 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction test 2026-02-20T17:56:19.947 INFO:teuthology.orchestra.run.trial195.stdout:Transaction test succeeded. 2026-02-20T17:56:19.947 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction 2026-02-20T17:56:20.279 INFO:teuthology.orchestra.run.trial195.stdout: Preparing : 1/1 2026-02-20T17:56:20.292 INFO:teuthology.orchestra.run.trial195.stdout: Installing : thrift-0.15.0-4.el9.x86_64 1/36 2026-02-20T17:56:20.445 INFO:teuthology.orchestra.run.trial195.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 2/36 2026-02-20T17:56:20.450 INFO:teuthology.orchestra.run.trial195.stdout: Upgrading : librados2-2:20.2.0-677.gf7870454.el9.x86_64 3/36 2026-02-20T17:56:20.480 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: librados2-2:20.2.0-677.gf7870454.el9.x86_64 3/36 2026-02-20T17:56:20.517 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-rados-2:20.2.0-677.gf7870454.el9.x86_64 4/36 2026-02-20T17:56:20.518 INFO:teuthology.orchestra.run.trial195.stdout: Installing : libcephfs2-2:20.2.0-677.gf7870454.el9.x86_64 5/36 2026-02-20T17:56:20.531 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: libcephfs2-2:20.2.0-677.gf7870454.el9.x86_64 5/36 2026-02-20T17:56:20.540 INFO:teuthology.orchestra.run.trial195.stdout: Installing : liboath-2.6.12-1.el9.x86_64 6/36 2026-02-20T17:56:20.543 INFO:teuthology.orchestra.run.trial195.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 7/36 2026-02-20T17:56:20.546 INFO:teuthology.orchestra.run.trial195.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 8/36 2026-02-20T17:56:20.550 INFO:teuthology.orchestra.run.trial195.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 9/36 2026-02-20T17:56:20.559 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-ceph-argparse-2:20.2.0-677.gf7870454.el9.x 10/36 2026-02-20T17:56:20.571 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-cephfs-2:20.2.0-677.gf7870454.el9.x86_64 11/36 2026-02-20T17:56:20.571 INFO:teuthology.orchestra.run.trial195.stdout: Installing : libradosstriper1-2:20.2.0-677.gf7870454.el9.x86_64 12/36 2026-02-20T17:56:20.584 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: libradosstriper1-2:20.2.0-677.gf7870454.el9.x86_64 12/36 2026-02-20T17:56:20.616 INFO:teuthology.orchestra.run.trial195.stdout: Installing : re2-1:20211101-20.el9.x86_64 13/36 2026-02-20T17:56:20.619 INFO:teuthology.orchestra.run.trial195.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 14/36 2026-02-20T17:56:20.626 INFO:teuthology.orchestra.run.trial195.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 15/36 2026-02-20T17:56:20.628 INFO:teuthology.orchestra.run.trial195.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 16/36 2026-02-20T17:56:20.633 INFO:teuthology.orchestra.run.trial195.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 17/36 2026-02-20T17:56:20.756 INFO:teuthology.orchestra.run.trial195.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 18/36 2026-02-20T17:56:20.757 INFO:teuthology.orchestra.run.trial195.stdout: Upgrading : librbd1-2:20.2.0-677.gf7870454.el9.x86_64 19/36 2026-02-20T17:56:20.782 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: librbd1-2:20.2.0-677.gf7870454.el9.x86_64 19/36 2026-02-20T17:56:20.784 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-rbd-2:20.2.0-677.gf7870454.el9.x86_64 20/36 2026-02-20T17:56:20.790 INFO:teuthology.orchestra.run.trial195.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 21/36 2026-02-20T17:56:21.070 INFO:teuthology.orchestra.run.trial195.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 22/36 2026-02-20T17:56:21.071 INFO:teuthology.orchestra.run.trial195.stdout: Installing : librgw2-2:20.2.0-677.gf7870454.el9.x86_64 23/36 2026-02-20T17:56:21.087 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: librgw2-2:20.2.0-677.gf7870454.el9.x86_64 23/36 2026-02-20T17:56:21.091 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-rgw-2:20.2.0-677.gf7870454.el9.x86_64 24/36 2026-02-20T17:56:21.096 INFO:teuthology.orchestra.run.trial195.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 25/36 2026-02-20T17:56:21.099 INFO:teuthology.orchestra.run.trial195.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 26/36 2026-02-20T17:56:21.100 INFO:teuthology.orchestra.run.trial195.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 27/36 2026-02-20T17:56:21.118 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 27/36 2026-02-20T17:56:21.121 INFO:teuthology.orchestra.run.trial195.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 28/36 2026-02-20T17:56:21.155 INFO:teuthology.orchestra.run.trial195.stdout: Installing : mailcap-2.1.49-5.el9.noarch 29/36 2026-02-20T17:56:21.156 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-ceph-common-2:20.2.0-677.gf7870454.el9.x86 30/36 2026-02-20T17:56:22.233 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: ceph-common-2:20.2.0-677.gf7870454.el9.x86_64 31/36 2026-02-20T17:56:22.235 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-common-2:20.2.0-677.gf7870454.el9.x86_64 31/36 2026-02-20T17:56:22.532 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: ceph-common-2:20.2.0-677.gf7870454.el9.x86_64 31/36 2026-02-20T17:56:22.535 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-base-2:20.2.0-677.gf7870454.el9.x86_64 32/36 2026-02-20T17:56:22.577 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: ceph-base-2:20.2.0-677.gf7870454.el9.x86_64 32/36 2026-02-20T17:56:22.578 INFO:teuthology.orchestra.run.trial195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-20T17:56:22.578 INFO:teuthology.orchestra.run.trial195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-20T17:56:22.578 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:22.581 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-selinux-2:20.2.0-677.gf7870454.el9.x86_64 33/36 2026-02-20T17:56:27.114 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: ceph-selinux-2:20.2.0-677.gf7870454.el9.x86_64 33/36 2026-02-20T17:56:27.114 INFO:teuthology.orchestra.run.trial195.stdout:skipping the directory /sys 2026-02-20T17:56:27.114 INFO:teuthology.orchestra.run.trial195.stdout:skipping the directory /proc 2026-02-20T17:56:27.115 INFO:teuthology.orchestra.run.trial195.stdout:skipping the directory /mnt 2026-02-20T17:56:27.115 INFO:teuthology.orchestra.run.trial195.stdout:skipping the directory /var/tmp 2026-02-20T17:56:27.115 INFO:teuthology.orchestra.run.trial195.stdout:skipping the directory /home 2026-02-20T17:56:27.115 INFO:teuthology.orchestra.run.trial195.stdout:skipping the directory /root 2026-02-20T17:56:27.115 INFO:teuthology.orchestra.run.trial195.stdout:skipping the directory /tmp 2026-02-20T17:56:27.115 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:28.139 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-radosgw-2:20.2.0-677.gf7870454.el9.x86_64 34/36 2026-02-20T17:56:28.162 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: ceph-radosgw-2:20.2.0-677.gf7870454.el9.x86_64 34/36 2026-02-20T17:56:28.162 INFO:teuthology.orchestra.run.trial195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T17:56:28.162 INFO:teuthology.orchestra.run.trial195.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-20T17:56:28.163 INFO:teuthology.orchestra.run.trial195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-20T17:56:28.163 INFO:teuthology.orchestra.run.trial195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-20T17:56:28.163 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:28.164 INFO:teuthology.orchestra.run.trial195.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 35/36 2026-02-20T17:56:28.178 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 35/36 2026-02-20T17:56:28.178 INFO:teuthology.orchestra.run.trial195.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 36/36 2026-02-20T17:56:28.234 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 36/36 2026-02-20T17:56:28.235 INFO:teuthology.orchestra.run.trial195.stdout:Creating group 'qat' with GID 992. 2026-02-20T17:56:28.235 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:28.570 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-base-2:20.2.0-677.gf7870454.el9.x86_64 1/36 2026-02-20T17:56:28.571 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-common-2:20.2.0-677.gf7870454.el9.x86_64 2/36 2026-02-20T17:56:28.571 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-radosgw-2:20.2.0-677.gf7870454.el9.x86_64 3/36 2026-02-20T17:56:28.571 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-selinux-2:20.2.0-677.gf7870454.el9.x86_64 4/36 2026-02-20T17:56:28.571 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : libcephfs2-2:20.2.0-677.gf7870454.el9.x86_64 5/36 2026-02-20T17:56:28.571 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : libradosstriper1-2:20.2.0-677.gf7870454.el9.x86_64 6/36 2026-02-20T17:56:28.571 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : librgw2-2:20.2.0-677.gf7870454.el9.x86_64 7/36 2026-02-20T17:56:28.571 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-ceph-argparse-2:20.2.0-677.gf7870454.el9.x 8/36 2026-02-20T17:56:28.571 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-ceph-common-2:20.2.0-677.gf7870454.el9.x86 9/36 2026-02-20T17:56:28.571 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-cephfs-2:20.2.0-677.gf7870454.el9.x86_64 10/36 2026-02-20T17:56:28.571 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-rados-2:20.2.0-677.gf7870454.el9.x86_64 11/36 2026-02-20T17:56:28.572 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-rbd-2:20.2.0-677.gf7870454.el9.x86_64 12/36 2026-02-20T17:56:28.572 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-rgw-2:20.2.0-677.gf7870454.el9.x86_64 13/36 2026-02-20T17:56:28.572 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 14/36 2026-02-20T17:56:28.572 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 15/36 2026-02-20T17:56:28.572 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 16/36 2026-02-20T17:56:28.572 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 17/36 2026-02-20T17:56:28.572 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 18/36 2026-02-20T17:56:28.572 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 19/36 2026-02-20T17:56:28.572 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 20/36 2026-02-20T17:56:28.572 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 21/36 2026-02-20T17:56:28.572 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 22/36 2026-02-20T17:56:28.573 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 23/36 2026-02-20T17:56:28.573 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 24/36 2026-02-20T17:56:28.573 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 25/36 2026-02-20T17:56:28.573 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 26/36 2026-02-20T17:56:28.573 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 27/36 2026-02-20T17:56:28.573 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 28/36 2026-02-20T17:56:28.573 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 29/36 2026-02-20T17:56:28.573 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 30/36 2026-02-20T17:56:28.573 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : re2-1:20211101-20.el9.x86_64 31/36 2026-02-20T17:56:28.573 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 32/36 2026-02-20T17:56:28.573 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : librados2-2:20.2.0-677.gf7870454.el9.x86_64 33/36 2026-02-20T17:56:28.573 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 34/36 2026-02-20T17:56:28.573 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : librbd1-2:20.2.0-677.gf7870454.el9.x86_64 35/36 2026-02-20T17:56:28.631 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 36/36 2026-02-20T17:56:28.631 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:28.631 INFO:teuthology.orchestra.run.trial195.stdout:Upgraded: 2026-02-20T17:56:28.631 INFO:teuthology.orchestra.run.trial195.stdout: librados2-2:20.2.0-677.gf7870454.el9.x86_64 2026-02-20T17:56:28.631 INFO:teuthology.orchestra.run.trial195.stdout: librbd1-2:20.2.0-677.gf7870454.el9.x86_64 2026-02-20T17:56:28.631 INFO:teuthology.orchestra.run.trial195.stdout:Installed: 2026-02-20T17:56:28.631 INFO:teuthology.orchestra.run.trial195.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-20T17:56:28.631 INFO:teuthology.orchestra.run.trial195.stdout: ceph-base-2:20.2.0-677.gf7870454.el9.x86_64 2026-02-20T17:56:28.632 INFO:teuthology.orchestra.run.trial195.stdout: ceph-common-2:20.2.0-677.gf7870454.el9.x86_64 2026-02-20T17:56:28.632 INFO:teuthology.orchestra.run.trial195.stdout: ceph-radosgw-2:20.2.0-677.gf7870454.el9.x86_64 2026-02-20T17:56:28.632 INFO:teuthology.orchestra.run.trial195.stdout: ceph-selinux-2:20.2.0-677.gf7870454.el9.x86_64 2026-02-20T17:56:28.632 INFO:teuthology.orchestra.run.trial195.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-20T17:56:28.632 INFO:teuthology.orchestra.run.trial195.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-20T17:56:28.632 INFO:teuthology.orchestra.run.trial195.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-20T17:56:28.632 INFO:teuthology.orchestra.run.trial195.stdout: libcephfs2-2:20.2.0-677.gf7870454.el9.x86_64 2026-02-20T17:56:28.632 INFO:teuthology.orchestra.run.trial195.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-20T17:56:28.632 INFO:teuthology.orchestra.run.trial195.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-20T17:56:28.632 INFO:teuthology.orchestra.run.trial195.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-20T17:56:28.633 INFO:teuthology.orchestra.run.trial195.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-20T17:56:28.633 INFO:teuthology.orchestra.run.trial195.stdout: libradosstriper1-2:20.2.0-677.gf7870454.el9.x86_64 2026-02-20T17:56:28.633 INFO:teuthology.orchestra.run.trial195.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-20T17:56:28.633 INFO:teuthology.orchestra.run.trial195.stdout: librgw2-2:20.2.0-677.gf7870454.el9.x86_64 2026-02-20T17:56:28.633 INFO:teuthology.orchestra.run.trial195.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-20T17:56:28.633 INFO:teuthology.orchestra.run.trial195.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-20T17:56:28.633 INFO:teuthology.orchestra.run.trial195.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-20T17:56:28.633 INFO:teuthology.orchestra.run.trial195.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-20T17:56:28.633 INFO:teuthology.orchestra.run.trial195.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-20T17:56:28.633 INFO:teuthology.orchestra.run.trial195.stdout: python3-ceph-argparse-2:20.2.0-677.gf7870454.el9.x86_64 2026-02-20T17:56:28.634 INFO:teuthology.orchestra.run.trial195.stdout: python3-ceph-common-2:20.2.0-677.gf7870454.el9.x86_64 2026-02-20T17:56:28.634 INFO:teuthology.orchestra.run.trial195.stdout: python3-cephfs-2:20.2.0-677.gf7870454.el9.x86_64 2026-02-20T17:56:28.634 INFO:teuthology.orchestra.run.trial195.stdout: python3-rados-2:20.2.0-677.gf7870454.el9.x86_64 2026-02-20T17:56:28.634 INFO:teuthology.orchestra.run.trial195.stdout: python3-rbd-2:20.2.0-677.gf7870454.el9.x86_64 2026-02-20T17:56:28.634 INFO:teuthology.orchestra.run.trial195.stdout: python3-rgw-2:20.2.0-677.gf7870454.el9.x86_64 2026-02-20T17:56:28.634 INFO:teuthology.orchestra.run.trial195.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-20T17:56:28.634 INFO:teuthology.orchestra.run.trial195.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-20T17:56:28.634 INFO:teuthology.orchestra.run.trial195.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-20T17:56:28.634 INFO:teuthology.orchestra.run.trial195.stdout: re2-1:20211101-20.el9.x86_64 2026-02-20T17:56:28.634 INFO:teuthology.orchestra.run.trial195.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-20T17:56:28.635 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:28.635 INFO:teuthology.orchestra.run.trial195.stdout:Complete! 2026-02-20T17:56:28.673 DEBUG:teuthology.orchestra.run.trial195:> sudo yum -y install ceph-test 2026-02-20T17:56:28.985 INFO:teuthology.orchestra.run.trial195.stdout:Last metadata expiration check: 0:00:14 ago on Fri 20 Feb 2026 05:56:14 PM UTC. 2026-02-20T17:56:29.068 INFO:teuthology.orchestra.run.trial195.stdout:Dependencies resolved. 2026-02-20T17:56:29.068 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:56:29.068 INFO:teuthology.orchestra.run.trial195.stdout: Package Arch Version Repository Size 2026-02-20T17:56:29.068 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:56:29.068 INFO:teuthology.orchestra.run.trial195.stdout:Installing: 2026-02-20T17:56:29.068 INFO:teuthology.orchestra.run.trial195.stdout: ceph-test x86_64 2:20.2.0-677.gf7870454.el9 ceph 84 M 2026-02-20T17:56:29.068 INFO:teuthology.orchestra.run.trial195.stdout:Installing dependencies: 2026-02-20T17:56:29.068 INFO:teuthology.orchestra.run.trial195.stdout: libcephsqlite x86_64 2:20.2.0-677.gf7870454.el9 ceph 164 k 2026-02-20T17:56:29.068 INFO:teuthology.orchestra.run.trial195.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-20T17:56:29.068 INFO:teuthology.orchestra.run.trial195.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-20T17:56:29.068 INFO:teuthology.orchestra.run.trial195.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-20T17:56:29.068 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:29.068 INFO:teuthology.orchestra.run.trial195.stdout:Transaction Summary 2026-02-20T17:56:29.068 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:56:29.068 INFO:teuthology.orchestra.run.trial195.stdout:Install 5 Packages 2026-02-20T17:56:29.068 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:29.069 INFO:teuthology.orchestra.run.trial195.stdout:Total download size: 85 M 2026-02-20T17:56:29.069 INFO:teuthology.orchestra.run.trial195.stdout:Installed size: 365 M 2026-02-20T17:56:29.069 INFO:teuthology.orchestra.run.trial195.stdout:Downloading Packages: 2026-02-20T17:56:29.177 INFO:teuthology.orchestra.run.trial195.stdout:(1/5): libxslt-1.1.34-12.el9.x86_64.rpm 4.7 MB/s | 233 kB 00:00 2026-02-20T17:56:29.197 INFO:teuthology.orchestra.run.trial195.stdout:(2/5): socat-1.7.4.1-8.el9.x86_64.rpm 16 MB/s | 303 kB 00:00 2026-02-20T17:56:29.206 INFO:teuthology.orchestra.run.trial195.stdout:(3/5): xmlstarlet-1.6.1-20.el9.x86_64.rpm 6.7 MB/s | 64 kB 00:00 2026-02-20T17:56:29.212 INFO:teuthology.orchestra.run.trial195.stdout:(4/5): libcephsqlite-20.2.0-677.gf7870454.el9.x 2.0 MB/s | 164 kB 00:00 2026-02-20T17:56:30.780 INFO:teuthology.orchestra.run.trial195.stdout:(5/5): ceph-test-20.2.0-677.gf7870454.el9.x86_6 51 MB/s | 84 MB 00:01 2026-02-20T17:56:30.781 INFO:teuthology.orchestra.run.trial195.stdout:-------------------------------------------------------------------------------- 2026-02-20T17:56:30.782 INFO:teuthology.orchestra.run.trial195.stdout:Total 49 MB/s | 85 MB 00:01 2026-02-20T17:56:30.807 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction check 2026-02-20T17:56:30.816 INFO:teuthology.orchestra.run.trial195.stdout:Transaction check succeeded. 2026-02-20T17:56:30.816 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction test 2026-02-20T17:56:31.104 INFO:teuthology.orchestra.run.trial195.stdout:Transaction test succeeded. 2026-02-20T17:56:31.104 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction 2026-02-20T17:56:31.419 INFO:teuthology.orchestra.run.trial195.stdout: Preparing : 1/1 2026-02-20T17:56:31.425 INFO:teuthology.orchestra.run.trial195.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 1/5 2026-02-20T17:56:31.429 INFO:teuthology.orchestra.run.trial195.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 2/5 2026-02-20T17:56:31.437 INFO:teuthology.orchestra.run.trial195.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/5 2026-02-20T17:56:31.438 INFO:teuthology.orchestra.run.trial195.stdout: Installing : libcephsqlite-2:20.2.0-677.gf7870454.el9.x86_64 4/5 2026-02-20T17:56:35.239 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: libcephsqlite-2:20.2.0-677.gf7870454.el9.x86_64 4/5 2026-02-20T17:56:35.257 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-test-2:20.2.0-677.gf7870454.el9.x86_64 5/5 2026-02-20T17:56:35.605 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: ceph-test-2:20.2.0-677.gf7870454.el9.x86_64 5/5 2026-02-20T17:56:35.606 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-test-2:20.2.0-677.gf7870454.el9.x86_64 1/5 2026-02-20T17:56:35.606 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : libcephsqlite-2:20.2.0-677.gf7870454.el9.x86_64 2/5 2026-02-20T17:56:35.606 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 3/5 2026-02-20T17:56:35.606 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 4/5 2026-02-20T17:56:35.660 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/5 2026-02-20T17:56:35.661 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:35.661 INFO:teuthology.orchestra.run.trial195.stdout:Installed: 2026-02-20T17:56:35.661 INFO:teuthology.orchestra.run.trial195.stdout: ceph-test-2:20.2.0-677.gf7870454.el9.x86_64 2026-02-20T17:56:35.661 INFO:teuthology.orchestra.run.trial195.stdout: libcephsqlite-2:20.2.0-677.gf7870454.el9.x86_64 2026-02-20T17:56:35.661 INFO:teuthology.orchestra.run.trial195.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-20T17:56:35.661 INFO:teuthology.orchestra.run.trial195.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-20T17:56:35.661 INFO:teuthology.orchestra.run.trial195.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-20T17:56:35.661 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:35.661 INFO:teuthology.orchestra.run.trial195.stdout:Complete! 2026-02-20T17:56:35.699 DEBUG:teuthology.orchestra.run.trial195:> sudo yum -y install ceph 2026-02-20T17:56:36.007 INFO:teuthology.orchestra.run.trial195.stdout:Last metadata expiration check: 0:00:22 ago on Fri 20 Feb 2026 05:56:14 PM UTC. 2026-02-20T17:56:36.090 INFO:teuthology.orchestra.run.trial195.stdout:Dependencies resolved. 2026-02-20T17:56:36.092 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:56:36.092 INFO:teuthology.orchestra.run.trial195.stdout: Package Arch Version Repository Size 2026-02-20T17:56:36.092 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:56:36.092 INFO:teuthology.orchestra.run.trial195.stdout:Installing: 2026-02-20T17:56:36.092 INFO:teuthology.orchestra.run.trial195.stdout: ceph x86_64 2:20.2.0-677.gf7870454.el9 ceph 6.5 k 2026-02-20T17:56:36.092 INFO:teuthology.orchestra.run.trial195.stdout:Installing dependencies: 2026-02-20T17:56:36.092 INFO:teuthology.orchestra.run.trial195.stdout: ceph-mds x86_64 2:20.2.0-677.gf7870454.el9 ceph 2.3 M 2026-02-20T17:56:36.092 INFO:teuthology.orchestra.run.trial195.stdout: ceph-mgr x86_64 2:20.2.0-677.gf7870454.el9 ceph 961 k 2026-02-20T17:56:36.093 INFO:teuthology.orchestra.run.trial195.stdout: ceph-mgr-modules-core noarch 2:20.2.0-677.gf7870454.el9 ceph-noarch 290 k 2026-02-20T17:56:36.093 INFO:teuthology.orchestra.run.trial195.stdout: ceph-mon x86_64 2:20.2.0-677.gf7870454.el9 ceph 5.0 M 2026-02-20T17:56:36.093 INFO:teuthology.orchestra.run.trial195.stdout: ceph-osd x86_64 2:20.2.0-677.gf7870454.el9 ceph 17 M 2026-02-20T17:56:36.093 INFO:teuthology.orchestra.run.trial195.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-20T17:56:36.093 INFO:teuthology.orchestra.run.trial195.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-20T17:56:36.093 INFO:teuthology.orchestra.run.trial195.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-20T17:56:36.093 INFO:teuthology.orchestra.run.trial195.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-20T17:56:36.093 INFO:teuthology.orchestra.run.trial195.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-20T17:56:36.093 INFO:teuthology.orchestra.run.trial195.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-20T17:56:36.093 INFO:teuthology.orchestra.run.trial195.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-20T17:56:36.093 INFO:teuthology.orchestra.run.trial195.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-20T17:56:36.093 INFO:teuthology.orchestra.run.trial195.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-20T17:56:36.093 INFO:teuthology.orchestra.run.trial195.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-20T17:56:36.094 INFO:teuthology.orchestra.run.trial195.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-20T17:56:36.094 INFO:teuthology.orchestra.run.trial195.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-20T17:56:36.094 INFO:teuthology.orchestra.run.trial195.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-20T17:56:36.094 INFO:teuthology.orchestra.run.trial195.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-20T17:56:36.094 INFO:teuthology.orchestra.run.trial195.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-20T17:56:36.094 INFO:teuthology.orchestra.run.trial195.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-20T17:56:36.094 INFO:teuthology.orchestra.run.trial195.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-20T17:56:36.094 INFO:teuthology.orchestra.run.trial195.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-20T17:56:36.094 INFO:teuthology.orchestra.run.trial195.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-20T17:56:36.094 INFO:teuthology.orchestra.run.trial195.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-20T17:56:36.094 INFO:teuthology.orchestra.run.trial195.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-20T17:56:36.094 INFO:teuthology.orchestra.run.trial195.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-20T17:56:36.095 INFO:teuthology.orchestra.run.trial195.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-20T17:56:36.095 INFO:teuthology.orchestra.run.trial195.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-20T17:56:36.095 INFO:teuthology.orchestra.run.trial195.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-20T17:56:36.095 INFO:teuthology.orchestra.run.trial195.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-20T17:56:36.095 INFO:teuthology.orchestra.run.trial195.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-20T17:56:36.095 INFO:teuthology.orchestra.run.trial195.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-20T17:56:36.095 INFO:teuthology.orchestra.run.trial195.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-20T17:56:36.095 INFO:teuthology.orchestra.run.trial195.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-20T17:56:36.095 INFO:teuthology.orchestra.run.trial195.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-20T17:56:36.095 INFO:teuthology.orchestra.run.trial195.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-20T17:56:36.095 INFO:teuthology.orchestra.run.trial195.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-20T17:56:36.095 INFO:teuthology.orchestra.run.trial195.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-20T17:56:36.096 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:36.096 INFO:teuthology.orchestra.run.trial195.stdout:Transaction Summary 2026-02-20T17:56:36.096 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:56:36.096 INFO:teuthology.orchestra.run.trial195.stdout:Install 40 Packages 2026-02-20T17:56:36.096 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:36.097 INFO:teuthology.orchestra.run.trial195.stdout:Total download size: 30 M 2026-02-20T17:56:36.097 INFO:teuthology.orchestra.run.trial195.stdout:Installed size: 100 M 2026-02-20T17:56:36.097 INFO:teuthology.orchestra.run.trial195.stdout:Downloading Packages: 2026-02-20T17:56:36.666 INFO:teuthology.orchestra.run.trial195.stdout:(1/40): ceph-20.2.0-677.gf7870454.el9.x86_64.rp 121 kB/s | 6.5 kB 00:00 2026-02-20T17:56:36.729 INFO:teuthology.orchestra.run.trial195.stdout:(2/40): ceph-mgr-20.2.0-677.gf7870454.el9.x86_6 8.1 MB/s | 961 kB 00:00 2026-02-20T17:56:36.786 INFO:teuthology.orchestra.run.trial195.stdout:(3/40): ceph-mds-20.2.0-677.gf7870454.el9.x86_6 13 MB/s | 2.3 MB 00:00 2026-02-20T17:56:36.827 INFO:teuthology.orchestra.run.trial195.stdout:(4/40): ceph-mgr-modules-core-20.2.0-677.gf7870 7.0 MB/s | 290 kB 00:00 2026-02-20T17:56:36.856 INFO:teuthology.orchestra.run.trial195.stdout:(5/40): ledmon-libs-1.1.0-3.el9.x86_64.rpm 1.4 MB/s | 40 kB 00:00 2026-02-20T17:56:36.872 INFO:teuthology.orchestra.run.trial195.stdout:(6/40): libconfig-1.7.2-9.el9.x86_64.rpm 4.3 MB/s | 72 kB 00:00 2026-02-20T17:56:36.892 INFO:teuthology.orchestra.run.trial195.stdout:(7/40): python3-cffi-1.14.5-5.el9.x86_64.rpm 13 MB/s | 253 kB 00:00 2026-02-20T17:56:36.928 INFO:teuthology.orchestra.run.trial195.stdout:(8/40): python3-cryptography-36.0.1-5.el9.x86_6 35 MB/s | 1.2 MB 00:00 2026-02-20T17:56:36.935 INFO:teuthology.orchestra.run.trial195.stdout:(9/40): ceph-mon-20.2.0-677.gf7870454.el9.x86_6 19 MB/s | 5.0 MB 00:00 2026-02-20T17:56:36.938 INFO:teuthology.orchestra.run.trial195.stdout:(10/40): python3-ply-3.11-14.el9.noarch.rpm 10 MB/s | 106 kB 00:00 2026-02-20T17:56:36.949 INFO:teuthology.orchestra.run.trial195.stdout:(11/40): python3-pyparsing-2.4.7-9.el9.noarch.r 14 MB/s | 150 kB 00:00 2026-02-20T17:56:36.960 INFO:teuthology.orchestra.run.trial195.stdout:(12/40): python3-requests-2.25.1-10.el9.noarch. 11 MB/s | 126 kB 00:00 2026-02-20T17:56:36.972 INFO:teuthology.orchestra.run.trial195.stdout:(13/40): python3-urllib3-1.26.5-7.el9.noarch.rp 19 MB/s | 218 kB 00:00 2026-02-20T17:56:36.979 INFO:teuthology.orchestra.run.trial195.stdout:(14/40): python3-pycparser-2.20-6.el9.noarch.rp 3.1 MB/s | 135 kB 00:00 2026-02-20T17:56:36.983 INFO:teuthology.orchestra.run.trial195.stdout:(15/40): unzip-6.0-59.el9.x86_64.rpm 16 MB/s | 182 kB 00:00 2026-02-20T17:56:36.998 INFO:teuthology.orchestra.run.trial195.stdout:(16/40): zip-3.0-35.el9.x86_64.rpm 13 MB/s | 266 kB 00:00 2026-02-20T17:56:37.047 INFO:teuthology.orchestra.run.trial195.stdout:(17/40): lua-5.4.4-4.el9.x86_64.rpm 3.8 MB/s | 188 kB 00:00 2026-02-20T17:56:37.049 INFO:teuthology.orchestra.run.trial195.stdout:(18/40): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 3.7 MB/s | 246 kB 00:00 2026-02-20T17:56:37.060 INFO:teuthology.orchestra.run.trial195.stdout:(19/40): python3-packaging-20.9-5.el9.noarch.rp 7.1 MB/s | 77 kB 00:00 2026-02-20T17:56:37.061 INFO:teuthology.orchestra.run.trial195.stdout:(20/40): python3-libstoragemgmt-1.10.1-1.el9.x8 13 MB/s | 177 kB 00:00 2026-02-20T17:56:37.069 INFO:teuthology.orchestra.run.trial195.stdout:(21/40): python3-toml-0.10.2-6.el9.noarch.rpm 4.2 MB/s | 42 kB 00:00 2026-02-20T17:56:37.154 INFO:teuthology.orchestra.run.trial195.stdout:(22/40): ceph-osd-20.2.0-677.gf7870454.el9.x86_ 40 MB/s | 17 MB 00:00 2026-02-20T17:56:37.317 INFO:teuthology.orchestra.run.trial195.stdout:(23/40): lua-devel-5.4.4-4.el9.x86_64.rpm 87 kB/s | 22 kB 00:00 2026-02-20T17:56:37.369 INFO:teuthology.orchestra.run.trial195.stdout:(24/40): python3-autocommand-2.2.2-8.el9.noarch 137 kB/s | 29 kB 00:00 2026-02-20T17:56:37.443 INFO:teuthology.orchestra.run.trial195.stdout:(25/40): luarocks-3.9.2-5.el9.noarch.rpm 405 kB/s | 151 kB 00:00 2026-02-20T17:56:37.584 INFO:teuthology.orchestra.run.trial195.stdout:(26/40): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 202 kB/s | 43 kB 00:00 2026-02-20T17:56:37.594 INFO:teuthology.orchestra.run.trial195.stdout:(27/40): python3-backports-tarfile-1.2.0-1.el9. 217 kB/s | 60 kB 00:00 2026-02-20T17:56:37.733 INFO:teuthology.orchestra.run.trial195.stdout:(28/40): python3-jaraco-8.2.1-3.el9.noarch.rpm 77 kB/s | 11 kB 00:00 2026-02-20T17:56:37.791 INFO:teuthology.orchestra.run.trial195.stdout:(29/40): python3-cheroot-10.0.1-4.el9.noarch.rp 499 kB/s | 173 kB 00:00 2026-02-20T17:56:37.944 INFO:teuthology.orchestra.run.trial195.stdout:(30/40): python3-jaraco-classes-3.2.1-5.el9.noa 84 kB/s | 18 kB 00:00 2026-02-20T17:56:38.001 INFO:teuthology.orchestra.run.trial195.stdout:(31/40): python3-cherrypy-18.6.1-2.el9.noarch.r 860 kB/s | 358 kB 00:00 2026-02-20T17:56:38.002 INFO:teuthology.orchestra.run.trial195.stdout:(32/40): python3-jaraco-collections-3.0.0-8.el9 110 kB/s | 23 kB 00:00 2026-02-20T17:56:38.153 INFO:teuthology.orchestra.run.trial195.stdout:(33/40): python3-jaraco-context-6.0.1-3.el9.noa 94 kB/s | 20 kB 00:00 2026-02-20T17:56:38.210 INFO:teuthology.orchestra.run.trial195.stdout:(34/40): python3-jaraco-functools-3.5.0-2.el9.n 93 kB/s | 19 kB 00:00 2026-02-20T17:56:38.219 INFO:teuthology.orchestra.run.trial195.stdout:(35/40): python3-jaraco-text-4.0.0-2.el9.noarch 122 kB/s | 26 kB 00:00 2026-02-20T17:56:38.425 INFO:teuthology.orchestra.run.trial195.stdout:(36/40): python3-portend-3.1.0-2.el9.noarch.rpm 76 kB/s | 16 kB 00:00 2026-02-20T17:56:38.431 INFO:teuthology.orchestra.run.trial195.stdout:(37/40): python3-more-itertools-8.12.0-2.el9.no 283 kB/s | 79 kB 00:00 2026-02-20T17:56:38.496 INFO:teuthology.orchestra.run.trial195.stdout:(38/40): python3-pyOpenSSL-21.0.0-1.el9.noarch. 324 kB/s | 90 kB 00:00 2026-02-20T17:56:38.643 INFO:teuthology.orchestra.run.trial195.stdout:(39/40): python3-tempora-5.0.0-2.el9.noarch.rpm 164 kB/s | 36 kB 00:00 2026-02-20T17:56:38.648 INFO:teuthology.orchestra.run.trial195.stdout:(40/40): python3-zc-lockfile-2.0-10.el9.noarch. 92 kB/s | 20 kB 00:00 2026-02-20T17:56:38.652 INFO:teuthology.orchestra.run.trial195.stdout:-------------------------------------------------------------------------------- 2026-02-20T17:56:38.652 INFO:teuthology.orchestra.run.trial195.stdout:Total 12 MB/s | 30 MB 00:02 2026-02-20T17:56:38.791 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction check 2026-02-20T17:56:38.808 INFO:teuthology.orchestra.run.trial195.stdout:Transaction check succeeded. 2026-02-20T17:56:38.808 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction test 2026-02-20T17:56:38.952 INFO:teuthology.orchestra.run.trial195.stdout:Transaction test succeeded. 2026-02-20T17:56:38.952 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction 2026-02-20T17:56:39.144 INFO:teuthology.orchestra.run.trial195.stdout: Preparing : 1/1 2026-02-20T17:56:39.146 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/40 2026-02-20T17:56:39.150 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/40 2026-02-20T17:56:39.155 INFO:teuthology.orchestra.run.trial195.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/40 2026-02-20T17:56:39.157 INFO:teuthology.orchestra.run.trial195.stdout: Installing : unzip-6.0-59.el9.x86_64 4/40 2026-02-20T17:56:39.163 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 5/40 2026-02-20T17:56:39.187 INFO:teuthology.orchestra.run.trial195.stdout: Installing : zip-3.0-35.el9.x86_64 6/40 2026-02-20T17:56:39.192 INFO:teuthology.orchestra.run.trial195.stdout: Installing : luarocks-3.9.2-5.el9.noarch 7/40 2026-02-20T17:56:39.197 INFO:teuthology.orchestra.run.trial195.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 8/40 2026-02-20T17:56:39.201 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 9/40 2026-02-20T17:56:39.207 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 10/40 2026-02-20T17:56:39.211 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 11/40 2026-02-20T17:56:39.217 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 12/40 2026-02-20T17:56:39.222 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 13/40 2026-02-20T17:56:39.226 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 14/40 2026-02-20T17:56:39.231 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 15/40 2026-02-20T17:56:39.235 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 16/40 2026-02-20T17:56:39.242 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 17/40 2026-02-20T17:56:39.245 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 18/40 2026-02-20T17:56:39.268 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 19/40 2026-02-20T17:56:39.280 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 20/40 2026-02-20T17:56:39.285 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 21/40 2026-02-20T17:56:39.295 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 22/40 2026-02-20T17:56:39.301 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-packaging-20.9-5.el9.noarch 23/40 2026-02-20T17:56:39.315 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/40 2026-02-20T17:56:39.330 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/40 2026-02-20T17:56:39.405 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/40 2026-02-20T17:56:39.413 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 27/40 2026-02-20T17:56:39.436 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/40 2026-02-20T17:56:39.465 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 29/40 2026-02-20T17:56:39.471 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/40 2026-02-20T17:56:39.518 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/40 2026-02-20T17:56:39.561 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-mgr-modules-core-2:20.2.0-677.gf7870454.el9.n 32/40 2026-02-20T17:56:39.562 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-mgr-2:20.2.0-677.gf7870454.el9.x86_64 33/40 2026-02-20T17:56:39.584 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: ceph-mgr-2:20.2.0-677.gf7870454.el9.x86_64 33/40 2026-02-20T17:56:39.584 INFO:teuthology.orchestra.run.trial195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T17:56:39.584 INFO:teuthology.orchestra.run.trial195.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-20T17:56:39.585 INFO:teuthology.orchestra.run.trial195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-20T17:56:39.585 INFO:teuthology.orchestra.run.trial195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-20T17:56:39.585 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:39.588 INFO:teuthology.orchestra.run.trial195.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 34/40 2026-02-20T17:56:39.605 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 35/40 2026-02-20T17:56:39.606 INFO:teuthology.orchestra.run.trial195.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-20T17:56:39.606 INFO:teuthology.orchestra.run.trial195.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-20T17:56:39.606 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:39.614 INFO:teuthology.orchestra.run.trial195.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 35/40 2026-02-20T17:56:39.639 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 35/40 2026-02-20T17:56:39.639 INFO:teuthology.orchestra.run.trial195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-20T17:56:39.639 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:40.320 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 36/40 2026-02-20T17:56:40.322 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-osd-2:20.2.0-677.gf7870454.el9.x86_64 37/40 2026-02-20T17:56:40.347 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: ceph-osd-2:20.2.0-677.gf7870454.el9.x86_64 37/40 2026-02-20T17:56:40.347 INFO:teuthology.orchestra.run.trial195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T17:56:40.347 INFO:teuthology.orchestra.run.trial195.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-20T17:56:40.347 INFO:teuthology.orchestra.run.trial195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-20T17:56:40.348 INFO:teuthology.orchestra.run.trial195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-20T17:56:40.348 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:40.560 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-mon-2:20.2.0-677.gf7870454.el9.x86_64 38/40 2026-02-20T17:56:40.583 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: ceph-mon-2:20.2.0-677.gf7870454.el9.x86_64 38/40 2026-02-20T17:56:40.583 INFO:teuthology.orchestra.run.trial195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T17:56:40.583 INFO:teuthology.orchestra.run.trial195.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-20T17:56:40.583 INFO:teuthology.orchestra.run.trial195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-20T17:56:40.583 INFO:teuthology.orchestra.run.trial195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-20T17:56:40.583 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:40.678 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-mds-2:20.2.0-677.gf7870454.el9.x86_64 39/40 2026-02-20T17:56:40.698 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: ceph-mds-2:20.2.0-677.gf7870454.el9.x86_64 39/40 2026-02-20T17:56:40.698 INFO:teuthology.orchestra.run.trial195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T17:56:40.698 INFO:teuthology.orchestra.run.trial195.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-20T17:56:40.698 INFO:teuthology.orchestra.run.trial195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-20T17:56:40.698 INFO:teuthology.orchestra.run.trial195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-20T17:56:40.699 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:40.716 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-2:20.2.0-677.gf7870454.el9.x86_64 40/40 2026-02-20T17:56:41.051 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: ceph-2:20.2.0-677.gf7870454.el9.x86_64 40/40 2026-02-20T17:56:41.052 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-2:20.2.0-677.gf7870454.el9.x86_64 1/40 2026-02-20T17:56:41.052 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-mds-2:20.2.0-677.gf7870454.el9.x86_64 2/40 2026-02-20T17:56:41.052 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-mgr-2:20.2.0-677.gf7870454.el9.x86_64 3/40 2026-02-20T17:56:41.052 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-mon-2:20.2.0-677.gf7870454.el9.x86_64 4/40 2026-02-20T17:56:41.052 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-osd-2:20.2.0-677.gf7870454.el9.x86_64 5/40 2026-02-20T17:56:41.052 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-mgr-modules-core-2:20.2.0-677.gf7870454.el9.n 6/40 2026-02-20T17:56:41.052 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 7/40 2026-02-20T17:56:41.052 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 8/40 2026-02-20T17:56:41.052 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 9/40 2026-02-20T17:56:41.052 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 10/40 2026-02-20T17:56:41.053 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-ply-3.11-14.el9.noarch 11/40 2026-02-20T17:56:41.053 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 12/40 2026-02-20T17:56:41.053 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 13/40 2026-02-20T17:56:41.053 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 14/40 2026-02-20T17:56:41.053 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 15/40 2026-02-20T17:56:41.053 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : unzip-6.0-59.el9.x86_64 16/40 2026-02-20T17:56:41.053 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : zip-3.0-35.el9.x86_64 17/40 2026-02-20T17:56:41.053 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 18/40 2026-02-20T17:56:41.053 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : lua-5.4.4-4.el9.x86_64 19/40 2026-02-20T17:56:41.053 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 20/40 2026-02-20T17:56:41.053 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 21/40 2026-02-20T17:56:41.053 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 22/40 2026-02-20T17:56:41.054 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 23/40 2026-02-20T17:56:41.054 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 24/40 2026-02-20T17:56:41.054 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 25/40 2026-02-20T17:56:41.054 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 26/40 2026-02-20T17:56:41.054 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 27/40 2026-02-20T17:56:41.054 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 28/40 2026-02-20T17:56:41.054 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 29/40 2026-02-20T17:56:41.054 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 30/40 2026-02-20T17:56:41.054 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 31/40 2026-02-20T17:56:41.054 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 32/40 2026-02-20T17:56:41.054 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 33/40 2026-02-20T17:56:41.054 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 34/40 2026-02-20T17:56:41.054 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 35/40 2026-02-20T17:56:41.054 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 36/40 2026-02-20T17:56:41.055 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 37/40 2026-02-20T17:56:41.055 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 38/40 2026-02-20T17:56:41.055 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 39/40 2026-02-20T17:56:41.107 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 40/40 2026-02-20T17:56:41.107 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:41.107 INFO:teuthology.orchestra.run.trial195.stdout:Installed: 2026-02-20T17:56:41.108 INFO:teuthology.orchestra.run.trial195.stdout: ceph-2:20.2.0-677.gf7870454.el9.x86_64 2026-02-20T17:56:41.108 INFO:teuthology.orchestra.run.trial195.stdout: ceph-mds-2:20.2.0-677.gf7870454.el9.x86_64 2026-02-20T17:56:41.108 INFO:teuthology.orchestra.run.trial195.stdout: ceph-mgr-2:20.2.0-677.gf7870454.el9.x86_64 2026-02-20T17:56:41.108 INFO:teuthology.orchestra.run.trial195.stdout: ceph-mgr-modules-core-2:20.2.0-677.gf7870454.el9.noarch 2026-02-20T17:56:41.108 INFO:teuthology.orchestra.run.trial195.stdout: ceph-mon-2:20.2.0-677.gf7870454.el9.x86_64 2026-02-20T17:56:41.108 INFO:teuthology.orchestra.run.trial195.stdout: ceph-osd-2:20.2.0-677.gf7870454.el9.x86_64 2026-02-20T17:56:41.108 INFO:teuthology.orchestra.run.trial195.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-20T17:56:41.108 INFO:teuthology.orchestra.run.trial195.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-20T17:56:41.109 INFO:teuthology.orchestra.run.trial195.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-20T17:56:41.109 INFO:teuthology.orchestra.run.trial195.stdout: lua-5.4.4-4.el9.x86_64 2026-02-20T17:56:41.109 INFO:teuthology.orchestra.run.trial195.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-20T17:56:41.109 INFO:teuthology.orchestra.run.trial195.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-20T17:56:41.109 INFO:teuthology.orchestra.run.trial195.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-20T17:56:41.109 INFO:teuthology.orchestra.run.trial195.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-20T17:56:41.110 INFO:teuthology.orchestra.run.trial195.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-20T17:56:41.110 INFO:teuthology.orchestra.run.trial195.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-20T17:56:41.110 INFO:teuthology.orchestra.run.trial195.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-20T17:56:41.110 INFO:teuthology.orchestra.run.trial195.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-20T17:56:41.110 INFO:teuthology.orchestra.run.trial195.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-20T17:56:41.110 INFO:teuthology.orchestra.run.trial195.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-20T17:56:41.110 INFO:teuthology.orchestra.run.trial195.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-20T17:56:41.110 INFO:teuthology.orchestra.run.trial195.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-20T17:56:41.110 INFO:teuthology.orchestra.run.trial195.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-20T17:56:41.110 INFO:teuthology.orchestra.run.trial195.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-20T17:56:41.110 INFO:teuthology.orchestra.run.trial195.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-20T17:56:41.111 INFO:teuthology.orchestra.run.trial195.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-20T17:56:41.111 INFO:teuthology.orchestra.run.trial195.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-20T17:56:41.111 INFO:teuthology.orchestra.run.trial195.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-20T17:56:41.111 INFO:teuthology.orchestra.run.trial195.stdout: python3-ply-3.11-14.el9.noarch 2026-02-20T17:56:41.111 INFO:teuthology.orchestra.run.trial195.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-20T17:56:41.111 INFO:teuthology.orchestra.run.trial195.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-20T17:56:41.111 INFO:teuthology.orchestra.run.trial195.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-20T17:56:41.112 INFO:teuthology.orchestra.run.trial195.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-20T17:56:41.112 INFO:teuthology.orchestra.run.trial195.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-20T17:56:41.112 INFO:teuthology.orchestra.run.trial195.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-20T17:56:41.112 INFO:teuthology.orchestra.run.trial195.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-20T17:56:41.112 INFO:teuthology.orchestra.run.trial195.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-20T17:56:41.112 INFO:teuthology.orchestra.run.trial195.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-20T17:56:41.112 INFO:teuthology.orchestra.run.trial195.stdout: unzip-6.0-59.el9.x86_64 2026-02-20T17:56:41.112 INFO:teuthology.orchestra.run.trial195.stdout: zip-3.0-35.el9.x86_64 2026-02-20T17:56:41.112 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:41.112 INFO:teuthology.orchestra.run.trial195.stdout:Complete! 2026-02-20T17:56:41.144 DEBUG:teuthology.orchestra.run.trial195:> sudo yum -y install ceph-base 2026-02-20T17:56:41.456 INFO:teuthology.orchestra.run.trial195.stdout:Last metadata expiration check: 0:00:27 ago on Fri 20 Feb 2026 05:56:14 PM UTC. 2026-02-20T17:56:41.519 INFO:teuthology.orchestra.run.trial195.stdout:Package ceph-base-2:20.2.0-677.gf7870454.el9.x86_64 is already installed. 2026-02-20T17:56:41.537 INFO:teuthology.orchestra.run.trial195.stdout:Dependencies resolved. 2026-02-20T17:56:41.538 INFO:teuthology.orchestra.run.trial195.stdout:Nothing to do. 2026-02-20T17:56:41.538 INFO:teuthology.orchestra.run.trial195.stdout:Complete! 2026-02-20T17:56:41.558 DEBUG:teuthology.orchestra.run.trial195:> sudo yum -y install cephadm 2026-02-20T17:56:41.857 INFO:teuthology.orchestra.run.trial195.stdout:Last metadata expiration check: 0:00:27 ago on Fri 20 Feb 2026 05:56:14 PM UTC. 2026-02-20T17:56:41.938 INFO:teuthology.orchestra.run.trial195.stdout:Dependencies resolved. 2026-02-20T17:56:41.939 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:56:41.939 INFO:teuthology.orchestra.run.trial195.stdout: Package Arch Version Repository Size 2026-02-20T17:56:41.939 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:56:41.940 INFO:teuthology.orchestra.run.trial195.stdout:Installing: 2026-02-20T17:56:41.940 INFO:teuthology.orchestra.run.trial195.stdout: cephadm noarch 2:20.2.0-677.gf7870454.el9 ceph-noarch 1.0 M 2026-02-20T17:56:41.940 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:41.940 INFO:teuthology.orchestra.run.trial195.stdout:Transaction Summary 2026-02-20T17:56:41.940 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:56:41.940 INFO:teuthology.orchestra.run.trial195.stdout:Install 1 Package 2026-02-20T17:56:41.940 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:41.940 INFO:teuthology.orchestra.run.trial195.stdout:Total download size: 1.0 M 2026-02-20T17:56:41.940 INFO:teuthology.orchestra.run.trial195.stdout:Installed size: 1.0 M 2026-02-20T17:56:41.940 INFO:teuthology.orchestra.run.trial195.stdout:Downloading Packages: 2026-02-20T17:56:42.058 INFO:teuthology.orchestra.run.trial195.stdout:cephadm-20.2.0-677.gf7870454.el9.noarch.rpm 8.4 MB/s | 1.0 MB 00:00 2026-02-20T17:56:42.059 INFO:teuthology.orchestra.run.trial195.stdout:-------------------------------------------------------------------------------- 2026-02-20T17:56:42.059 INFO:teuthology.orchestra.run.trial195.stdout:Total 8.3 MB/s | 1.0 MB 00:00 2026-02-20T17:56:42.059 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction check 2026-02-20T17:56:42.061 INFO:teuthology.orchestra.run.trial195.stdout:Transaction check succeeded. 2026-02-20T17:56:42.062 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction test 2026-02-20T17:56:42.065 INFO:teuthology.orchestra.run.trial195.stdout:Transaction test succeeded. 2026-02-20T17:56:42.066 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction 2026-02-20T17:56:42.084 INFO:teuthology.orchestra.run.trial195.stdout: Preparing : 1/1 2026-02-20T17:56:42.145 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: cephadm-2:20.2.0-677.gf7870454.el9.noarch 1/1 2026-02-20T17:56:42.159 INFO:teuthology.orchestra.run.trial195.stdout: Installing : cephadm-2:20.2.0-677.gf7870454.el9.noarch 1/1 2026-02-20T17:56:42.222 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: cephadm-2:20.2.0-677.gf7870454.el9.noarch 1/1 2026-02-20T17:56:42.276 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : cephadm-2:20.2.0-677.gf7870454.el9.noarch 1/1 2026-02-20T17:56:42.276 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:42.276 INFO:teuthology.orchestra.run.trial195.stdout:Installed: 2026-02-20T17:56:42.277 INFO:teuthology.orchestra.run.trial195.stdout: cephadm-2:20.2.0-677.gf7870454.el9.noarch 2026-02-20T17:56:42.277 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:42.277 INFO:teuthology.orchestra.run.trial195.stdout:Complete! 2026-02-20T17:56:42.303 DEBUG:teuthology.orchestra.run.trial195:> sudo yum -y install ceph-immutable-object-cache 2026-02-20T17:56:42.618 INFO:teuthology.orchestra.run.trial195.stdout:Last metadata expiration check: 0:00:28 ago on Fri 20 Feb 2026 05:56:14 PM UTC. 2026-02-20T17:56:42.700 INFO:teuthology.orchestra.run.trial195.stdout:Dependencies resolved. 2026-02-20T17:56:42.701 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:56:42.701 INFO:teuthology.orchestra.run.trial195.stdout: Package Arch Version Repo Size 2026-02-20T17:56:42.701 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:56:42.701 INFO:teuthology.orchestra.run.trial195.stdout:Installing: 2026-02-20T17:56:42.701 INFO:teuthology.orchestra.run.trial195.stdout: ceph-immutable-object-cache x86_64 2:20.2.0-677.gf7870454.el9 ceph 154 k 2026-02-20T17:56:42.702 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:42.702 INFO:teuthology.orchestra.run.trial195.stdout:Transaction Summary 2026-02-20T17:56:42.702 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:56:42.702 INFO:teuthology.orchestra.run.trial195.stdout:Install 1 Package 2026-02-20T17:56:42.702 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:42.702 INFO:teuthology.orchestra.run.trial195.stdout:Total download size: 154 k 2026-02-20T17:56:42.702 INFO:teuthology.orchestra.run.trial195.stdout:Installed size: 447 k 2026-02-20T17:56:42.702 INFO:teuthology.orchestra.run.trial195.stdout:Downloading Packages: 2026-02-20T17:56:42.786 INFO:teuthology.orchestra.run.trial195.stdout:ceph-immutable-object-cache-20.2.0-677.gf787045 1.8 MB/s | 154 kB 00:00 2026-02-20T17:56:42.787 INFO:teuthology.orchestra.run.trial195.stdout:-------------------------------------------------------------------------------- 2026-02-20T17:56:42.787 INFO:teuthology.orchestra.run.trial195.stdout:Total 1.7 MB/s | 154 kB 00:00 2026-02-20T17:56:42.788 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction check 2026-02-20T17:56:42.792 INFO:teuthology.orchestra.run.trial195.stdout:Transaction check succeeded. 2026-02-20T17:56:42.792 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction test 2026-02-20T17:56:42.817 INFO:teuthology.orchestra.run.trial195.stdout:Transaction test succeeded. 2026-02-20T17:56:42.817 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction 2026-02-20T17:56:42.866 INFO:teuthology.orchestra.run.trial195.stdout: Preparing : 1/1 2026-02-20T17:56:42.868 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-immutable-object-cache-2:20.2.0-677.gf7870454.e 1/1 2026-02-20T17:56:42.891 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.2.0-677.gf7870454.e 1/1 2026-02-20T17:56:42.892 INFO:teuthology.orchestra.run.trial195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T17:56:42.892 INFO:teuthology.orchestra.run.trial195.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-20T17:56:42.892 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:43.170 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-immutable-object-cache-2:20.2.0-677.gf7870454.e 1/1 2026-02-20T17:56:43.170 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:43.170 INFO:teuthology.orchestra.run.trial195.stdout:Installed: 2026-02-20T17:56:43.170 INFO:teuthology.orchestra.run.trial195.stdout: ceph-immutable-object-cache-2:20.2.0-677.gf7870454.el9.x86_64 2026-02-20T17:56:43.171 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:43.171 INFO:teuthology.orchestra.run.trial195.stdout:Complete! 2026-02-20T17:56:43.198 DEBUG:teuthology.orchestra.run.trial195:> sudo yum -y install ceph-mgr 2026-02-20T17:56:43.505 INFO:teuthology.orchestra.run.trial195.stdout:Last metadata expiration check: 0:00:29 ago on Fri 20 Feb 2026 05:56:14 PM UTC. 2026-02-20T17:56:43.570 INFO:teuthology.orchestra.run.trial195.stdout:Package ceph-mgr-2:20.2.0-677.gf7870454.el9.x86_64 is already installed. 2026-02-20T17:56:43.588 INFO:teuthology.orchestra.run.trial195.stdout:Dependencies resolved. 2026-02-20T17:56:43.588 INFO:teuthology.orchestra.run.trial195.stdout:Nothing to do. 2026-02-20T17:56:43.589 INFO:teuthology.orchestra.run.trial195.stdout:Complete! 2026-02-20T17:56:43.610 DEBUG:teuthology.orchestra.run.trial195:> sudo yum -y install ceph-mgr-dashboard 2026-02-20T17:56:43.923 INFO:teuthology.orchestra.run.trial195.stdout:Last metadata expiration check: 0:00:29 ago on Fri 20 Feb 2026 05:56:14 PM UTC. 2026-02-20T17:56:44.006 INFO:teuthology.orchestra.run.trial195.stdout:Dependencies resolved. 2026-02-20T17:56:44.007 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:56:44.007 INFO:teuthology.orchestra.run.trial195.stdout: Package Arch Version Repository Size 2026-02-20T17:56:44.007 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:56:44.007 INFO:teuthology.orchestra.run.trial195.stdout:Installing: 2026-02-20T17:56:44.007 INFO:teuthology.orchestra.run.trial195.stdout: ceph-mgr-dashboard noarch 2:20.2.0-677.gf7870454.el9 ceph-noarch 11 M 2026-02-20T17:56:44.007 INFO:teuthology.orchestra.run.trial195.stdout:Installing dependencies: 2026-02-20T17:56:44.007 INFO:teuthology.orchestra.run.trial195.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-20T17:56:44.007 INFO:teuthology.orchestra.run.trial195.stdout: ceph-grafana-dashboards noarch 2:20.2.0-677.gf7870454.el9 ceph-noarch 43 k 2026-02-20T17:56:44.007 INFO:teuthology.orchestra.run.trial195.stdout: ceph-prometheus-alerts noarch 2:20.2.0-677.gf7870454.el9 ceph-noarch 17 k 2026-02-20T17:56:44.007 INFO:teuthology.orchestra.run.trial195.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-20T17:56:44.007 INFO:teuthology.orchestra.run.trial195.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-20T17:56:44.007 INFO:teuthology.orchestra.run.trial195.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-20T17:56:44.007 INFO:teuthology.orchestra.run.trial195.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-20T17:56:44.007 INFO:teuthology.orchestra.run.trial195.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-20T17:56:44.007 INFO:teuthology.orchestra.run.trial195.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-20T17:56:44.007 INFO:teuthology.orchestra.run.trial195.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-20T17:56:44.007 INFO:teuthology.orchestra.run.trial195.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-20T17:56:44.007 INFO:teuthology.orchestra.run.trial195.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-20T17:56:44.007 INFO:teuthology.orchestra.run.trial195.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-20T17:56:44.007 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:44.007 INFO:teuthology.orchestra.run.trial195.stdout:Transaction Summary 2026-02-20T17:56:44.007 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:56:44.007 INFO:teuthology.orchestra.run.trial195.stdout:Install 14 Packages 2026-02-20T17:56:44.007 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:44.008 INFO:teuthology.orchestra.run.trial195.stdout:Total download size: 16 M 2026-02-20T17:56:44.008 INFO:teuthology.orchestra.run.trial195.stdout:Installed size: 158 M 2026-02-20T17:56:44.008 INFO:teuthology.orchestra.run.trial195.stdout:Downloading Packages: 2026-02-20T17:56:44.624 INFO:teuthology.orchestra.run.trial195.stdout:(1/14): ceph-prometheus-alerts-20.2.0-677.gf787 254 kB/s | 17 kB 00:00 2026-02-20T17:56:44.628 INFO:teuthology.orchestra.run.trial195.stdout:(2/14): ceph-grafana-dashboards-20.2.0-677.gf78 600 kB/s | 43 kB 00:00 2026-02-20T17:56:44.692 INFO:teuthology.orchestra.run.trial195.stdout:(3/14): python3-jmespath-1.0.1-1.el9.noarch.rpm 747 kB/s | 48 kB 00:00 2026-02-20T17:56:44.757 INFO:teuthology.orchestra.run.trial195.stdout:(4/14): protobuf-3.14.0-17.el9.x86_64.rpm 7.6 MB/s | 1.0 MB 00:00 2026-02-20T17:56:44.758 INFO:teuthology.orchestra.run.trial195.stdout:(5/14): python3-protobuf-3.14.0-17.el9.noarch.r 4.0 MB/s | 267 kB 00:00 2026-02-20T17:56:44.820 INFO:teuthology.orchestra.run.trial195.stdout:(6/14): protobuf-compiler-3.14.0-17.el9.x86_64. 13 MB/s | 862 kB 00:00 2026-02-20T17:56:44.845 INFO:teuthology.orchestra.run.trial195.stdout:(7/14): ceph-mgr-dashboard-20.2.0-677.gf7870454 37 MB/s | 11 MB 00:00 2026-02-20T17:56:44.846 INFO:teuthology.orchestra.run.trial195.stdout:(8/14): abseil-cpp-20211102.0-4.el9.x86_64.rpm 6.1 MB/s | 551 kB 00:00 2026-02-20T17:56:44.847 INFO:teuthology.orchestra.run.trial195.stdout:(9/14): grpc-data-1.46.7-10.el9.noarch.rpm 706 kB/s | 19 kB 00:00 2026-02-20T17:56:44.862 INFO:teuthology.orchestra.run.trial195.stdout:(10/14): python3-repoze-lru-0.7-16.el9.noarch.r 2.1 MB/s | 31 kB 00:00 2026-02-20T17:56:44.878 INFO:teuthology.orchestra.run.trial195.stdout:(11/14): python3-grpcio-tools-1.46.7-10.el9.x86 4.4 MB/s | 144 kB 00:00 2026-02-20T17:56:44.893 INFO:teuthology.orchestra.run.trial195.stdout:(12/14): python3-xmltodict-0.12.0-15.el9.noarch 1.4 MB/s | 22 kB 00:00 2026-02-20T17:56:44.897 INFO:teuthology.orchestra.run.trial195.stdout:(13/14): python3-routes-2.5.1-5.el9.noarch.rpm 5.4 MB/s | 188 kB 00:00 2026-02-20T17:56:44.976 INFO:teuthology.orchestra.run.trial195.stdout:(14/14): python3-grpcio-1.46.7-10.el9.x86_64.rp 16 MB/s | 2.0 MB 00:00 2026-02-20T17:56:44.976 INFO:teuthology.orchestra.run.trial195.stdout:-------------------------------------------------------------------------------- 2026-02-20T17:56:44.977 INFO:teuthology.orchestra.run.trial195.stdout:Total 16 MB/s | 16 MB 00:00 2026-02-20T17:56:45.013 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction check 2026-02-20T17:56:45.021 INFO:teuthology.orchestra.run.trial195.stdout:Transaction check succeeded. 2026-02-20T17:56:45.021 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction test 2026-02-20T17:56:45.116 INFO:teuthology.orchestra.run.trial195.stdout:Transaction test succeeded. 2026-02-20T17:56:45.116 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction 2026-02-20T17:56:45.234 INFO:teuthology.orchestra.run.trial195.stdout: Preparing : 1/1 2026-02-20T17:56:45.243 INFO:teuthology.orchestra.run.trial195.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 1/14 2026-02-20T17:56:45.246 INFO:teuthology.orchestra.run.trial195.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 2/14 2026-02-20T17:56:45.251 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 3/14 2026-02-20T17:56:45.263 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 4/14 2026-02-20T17:56:45.265 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 5/14 2026-02-20T17:56:45.292 INFO:teuthology.orchestra.run.trial195.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 6/14 2026-02-20T17:56:45.335 INFO:teuthology.orchestra.run.trial195.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 7/14 2026-02-20T17:56:45.363 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 8/14 2026-02-20T17:56:45.373 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 9/14 2026-02-20T17:56:45.380 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/14 2026-02-20T17:56:45.383 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 11/14 2026-02-20T17:56:45.390 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-prometheus-alerts-2:20.2.0-677.gf7870454.el9. 12/14 2026-02-20T17:56:46.525 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-grafana-dashboards-2:20.2.0-677.gf7870454.el9 13/14 2026-02-20T17:56:46.530 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-mgr-dashboard-2:20.2.0-677.gf7870454.el9.noar 14/14 2026-02-20T17:56:46.724 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.2.0-677.gf7870454.el9.noar 14/14 2026-02-20T17:56:46.724 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-grafana-dashboards-2:20.2.0-677.gf7870454.el9 1/14 2026-02-20T17:56:46.725 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-mgr-dashboard-2:20.2.0-677.gf7870454.el9.noar 2/14 2026-02-20T17:56:46.725 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-prometheus-alerts-2:20.2.0-677.gf7870454.el9. 3/14 2026-02-20T17:56:46.725 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 4/14 2026-02-20T17:56:46.725 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 5/14 2026-02-20T17:56:46.725 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 6/14 2026-02-20T17:56:46.725 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 7/14 2026-02-20T17:56:46.725 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 8/14 2026-02-20T17:56:46.725 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 9/14 2026-02-20T17:56:46.725 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 10/14 2026-02-20T17:56:46.725 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 11/14 2026-02-20T17:56:46.725 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 12/14 2026-02-20T17:56:46.726 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 13/14 2026-02-20T17:56:46.778 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 14/14 2026-02-20T17:56:46.779 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:46.779 INFO:teuthology.orchestra.run.trial195.stdout:Installed: 2026-02-20T17:56:46.779 INFO:teuthology.orchestra.run.trial195.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-20T17:56:46.779 INFO:teuthology.orchestra.run.trial195.stdout: ceph-grafana-dashboards-2:20.2.0-677.gf7870454.el9.noarch 2026-02-20T17:56:46.779 INFO:teuthology.orchestra.run.trial195.stdout: ceph-mgr-dashboard-2:20.2.0-677.gf7870454.el9.noarch 2026-02-20T17:56:46.779 INFO:teuthology.orchestra.run.trial195.stdout: ceph-prometheus-alerts-2:20.2.0-677.gf7870454.el9.noarch 2026-02-20T17:56:46.779 INFO:teuthology.orchestra.run.trial195.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-20T17:56:46.779 INFO:teuthology.orchestra.run.trial195.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-20T17:56:46.779 INFO:teuthology.orchestra.run.trial195.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-20T17:56:46.779 INFO:teuthology.orchestra.run.trial195.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-20T17:56:46.780 INFO:teuthology.orchestra.run.trial195.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-20T17:56:46.780 INFO:teuthology.orchestra.run.trial195.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-20T17:56:46.780 INFO:teuthology.orchestra.run.trial195.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-20T17:56:46.780 INFO:teuthology.orchestra.run.trial195.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-20T17:56:46.780 INFO:teuthology.orchestra.run.trial195.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-20T17:56:46.780 INFO:teuthology.orchestra.run.trial195.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-20T17:56:46.780 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:46.780 INFO:teuthology.orchestra.run.trial195.stdout:Complete! 2026-02-20T17:56:46.812 DEBUG:teuthology.orchestra.run.trial195:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-20T17:56:47.122 INFO:teuthology.orchestra.run.trial195.stdout:Last metadata expiration check: 0:00:33 ago on Fri 20 Feb 2026 05:56:14 PM UTC. 2026-02-20T17:56:47.206 INFO:teuthology.orchestra.run.trial195.stdout:Dependencies resolved. 2026-02-20T17:56:47.207 INFO:teuthology.orchestra.run.trial195.stdout:====================================================================================== 2026-02-20T17:56:47.207 INFO:teuthology.orchestra.run.trial195.stdout: Package Arch Version Repository Size 2026-02-20T17:56:47.207 INFO:teuthology.orchestra.run.trial195.stdout:====================================================================================== 2026-02-20T17:56:47.207 INFO:teuthology.orchestra.run.trial195.stdout:Installing: 2026-02-20T17:56:47.208 INFO:teuthology.orchestra.run.trial195.stdout: ceph-mgr-diskprediction-local noarch 2:20.2.0-677.gf7870454.el9 ceph-noarch 7.4 M 2026-02-20T17:56:47.208 INFO:teuthology.orchestra.run.trial195.stdout:Installing dependencies: 2026-02-20T17:56:47.208 INFO:teuthology.orchestra.run.trial195.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-20T17:56:47.208 INFO:teuthology.orchestra.run.trial195.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-20T17:56:47.208 INFO:teuthology.orchestra.run.trial195.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-20T17:56:47.208 INFO:teuthology.orchestra.run.trial195.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-20T17:56:47.208 INFO:teuthology.orchestra.run.trial195.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-20T17:56:47.209 INFO:teuthology.orchestra.run.trial195.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-20T17:56:47.209 INFO:teuthology.orchestra.run.trial195.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-20T17:56:47.209 INFO:teuthology.orchestra.run.trial195.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-20T17:56:47.209 INFO:teuthology.orchestra.run.trial195.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-20T17:56:47.209 INFO:teuthology.orchestra.run.trial195.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-20T17:56:47.209 INFO:teuthology.orchestra.run.trial195.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-20T17:56:47.209 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:47.209 INFO:teuthology.orchestra.run.trial195.stdout:Transaction Summary 2026-02-20T17:56:47.209 INFO:teuthology.orchestra.run.trial195.stdout:====================================================================================== 2026-02-20T17:56:47.209 INFO:teuthology.orchestra.run.trial195.stdout:Install 12 Packages 2026-02-20T17:56:47.209 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:47.209 INFO:teuthology.orchestra.run.trial195.stdout:Total download size: 43 M 2026-02-20T17:56:47.210 INFO:teuthology.orchestra.run.trial195.stdout:Installed size: 234 M 2026-02-20T17:56:47.210 INFO:teuthology.orchestra.run.trial195.stdout:Downloading Packages: 2026-02-20T17:56:47.403 INFO:teuthology.orchestra.run.trial195.stdout:(1/12): libquadmath-11.5.0-14.el9.x86_64.rpm 3.7 MB/s | 184 kB 00:00 2026-02-20T17:56:47.414 INFO:teuthology.orchestra.run.trial195.stdout:(2/12): flexiblas-3.0.4-9.el9.x86_64.rpm 2.7 MB/s | 30 kB 00:00 2026-02-20T17:56:47.421 INFO:teuthology.orchestra.run.trial195.stdout:(3/12): libgfortran-11.5.0-14.el9.x86_64.rpm 11 MB/s | 794 kB 00:00 2026-02-20T17:56:47.432 INFO:teuthology.orchestra.run.trial195.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-9.el9.x 1.4 MB/s | 15 kB 00:00 2026-02-20T17:56:47.439 INFO:teuthology.orchestra.run.trial195.stdout:(5/12): openblas-0.3.29-1.el9.x86_64.rpm 5.7 MB/s | 42 kB 00:00 2026-02-20T17:56:47.442 INFO:teuthology.orchestra.run.trial195.stdout:(6/12): flexiblas-netlib-3.0.4-9.el9.x86_64.rpm 106 MB/s | 3.0 MB 00:00 2026-02-20T17:56:47.450 INFO:teuthology.orchestra.run.trial195.stdout:(7/12): python3-devel-3.9.25-3.el9.x86_64.rpm 33 MB/s | 244 kB 00:00 2026-02-20T17:56:47.476 INFO:teuthology.orchestra.run.trial195.stdout:(8/12): openblas-openmp-0.3.29-1.el9.x86_64.rpm 142 MB/s | 5.3 MB 00:00 2026-02-20T17:56:47.486 INFO:teuthology.orchestra.run.trial195.stdout:(9/12): python3-numpy-1.23.5-2.el9.x86_64.rpm 170 MB/s | 6.1 MB 00:00 2026-02-20T17:56:47.488 INFO:teuthology.orchestra.run.trial195.stdout:(10/12): python3-numpy-f2py-1.23.5-2.el9.x86_64 38 MB/s | 442 kB 00:00 2026-02-20T17:56:47.558 INFO:teuthology.orchestra.run.trial195.stdout:(11/12): python3-scipy-1.9.3-2.el9.x86_64.rpm 267 MB/s | 19 MB 00:00 2026-02-20T17:56:47.611 INFO:teuthology.orchestra.run.trial195.stdout:(12/12): ceph-mgr-diskprediction-local-20.2.0-6 29 MB/s | 7.4 MB 00:00 2026-02-20T17:56:47.612 INFO:teuthology.orchestra.run.trial195.stdout:-------------------------------------------------------------------------------- 2026-02-20T17:56:47.612 INFO:teuthology.orchestra.run.trial195.stdout:Total 106 MB/s | 43 MB 00:00 2026-02-20T17:56:47.772 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction check 2026-02-20T17:56:47.785 INFO:teuthology.orchestra.run.trial195.stdout:Transaction check succeeded. 2026-02-20T17:56:47.785 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction test 2026-02-20T17:56:47.985 INFO:teuthology.orchestra.run.trial195.stdout:Transaction test succeeded. 2026-02-20T17:56:47.985 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction 2026-02-20T17:56:48.201 INFO:teuthology.orchestra.run.trial195.stdout: Preparing : 1/1 2026-02-20T17:56:48.203 INFO:teuthology.orchestra.run.trial195.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 1/12 2026-02-20T17:56:48.209 INFO:teuthology.orchestra.run.trial195.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 2/12 2026-02-20T17:56:48.244 INFO:teuthology.orchestra.run.trial195.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 3/12 2026-02-20T17:56:48.248 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 4/12 2026-02-20T17:56:48.307 INFO:teuthology.orchestra.run.trial195.stdout: Installing : openblas-0.3.29-1.el9.x86_64 5/12 2026-02-20T17:56:48.309 INFO:teuthology.orchestra.run.trial195.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 6/12 2026-02-20T17:56:48.331 INFO:teuthology.orchestra.run.trial195.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 7/12 2026-02-20T17:56:48.657 INFO:teuthology.orchestra.run.trial195.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 8/12 2026-02-20T17:56:48.729 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 9/12 2026-02-20T17:56:49.401 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 10/12 2026-02-20T17:56:49.855 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 11/12 2026-02-20T17:56:49.856 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-mgr-diskprediction-local-2:20.2.0-677.gf78704 12/12 2026-02-20T17:56:50.134 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.2.0-677.gf78704 12/12 2026-02-20T17:56:50.134 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.2.0-677.gf78704 1/12 2026-02-20T17:56:50.134 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 2/12 2026-02-20T17:56:50.135 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 3/12 2026-02-20T17:56:50.135 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 4/12 2026-02-20T17:56:50.135 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 5/12 2026-02-20T17:56:50.135 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 6/12 2026-02-20T17:56:50.135 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 7/12 2026-02-20T17:56:50.135 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 8/12 2026-02-20T17:56:50.135 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 9/12 2026-02-20T17:56:50.135 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 10/12 2026-02-20T17:56:50.135 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 11/12 2026-02-20T17:56:50.193 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 12/12 2026-02-20T17:56:50.193 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:50.193 INFO:teuthology.orchestra.run.trial195.stdout:Installed: 2026-02-20T17:56:50.193 INFO:teuthology.orchestra.run.trial195.stdout: ceph-mgr-diskprediction-local-2:20.2.0-677.gf7870454.el9.noarch 2026-02-20T17:56:50.193 INFO:teuthology.orchestra.run.trial195.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-20T17:56:50.193 INFO:teuthology.orchestra.run.trial195.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-20T17:56:50.193 INFO:teuthology.orchestra.run.trial195.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-20T17:56:50.193 INFO:teuthology.orchestra.run.trial195.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-20T17:56:50.193 INFO:teuthology.orchestra.run.trial195.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-20T17:56:50.194 INFO:teuthology.orchestra.run.trial195.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-20T17:56:50.194 INFO:teuthology.orchestra.run.trial195.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-20T17:56:50.194 INFO:teuthology.orchestra.run.trial195.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-20T17:56:50.194 INFO:teuthology.orchestra.run.trial195.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-20T17:56:50.194 INFO:teuthology.orchestra.run.trial195.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-20T17:56:50.194 INFO:teuthology.orchestra.run.trial195.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-20T17:56:50.194 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:50.194 INFO:teuthology.orchestra.run.trial195.stdout:Complete! 2026-02-20T17:56:50.231 DEBUG:teuthology.orchestra.run.trial195:> sudo yum -y install ceph-mgr-rook 2026-02-20T17:56:50.552 INFO:teuthology.orchestra.run.trial195.stdout:Last metadata expiration check: 0:00:36 ago on Fri 20 Feb 2026 05:56:14 PM UTC. 2026-02-20T17:56:50.637 INFO:teuthology.orchestra.run.trial195.stdout:Dependencies resolved. 2026-02-20T17:56:50.638 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:56:50.638 INFO:teuthology.orchestra.run.trial195.stdout: Package Arch Version Repository Size 2026-02-20T17:56:50.638 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:56:50.639 INFO:teuthology.orchestra.run.trial195.stdout:Installing: 2026-02-20T17:56:50.639 INFO:teuthology.orchestra.run.trial195.stdout: ceph-mgr-rook noarch 2:20.2.0-677.gf7870454.el9 ceph-noarch 50 k 2026-02-20T17:56:50.639 INFO:teuthology.orchestra.run.trial195.stdout:Installing dependencies: 2026-02-20T17:56:50.639 INFO:teuthology.orchestra.run.trial195.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-20T17:56:50.639 INFO:teuthology.orchestra.run.trial195.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-20T17:56:50.639 INFO:teuthology.orchestra.run.trial195.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-20T17:56:50.639 INFO:teuthology.orchestra.run.trial195.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-20T17:56:50.639 INFO:teuthology.orchestra.run.trial195.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-20T17:56:50.639 INFO:teuthology.orchestra.run.trial195.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-20T17:56:50.639 INFO:teuthology.orchestra.run.trial195.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-20T17:56:50.639 INFO:teuthology.orchestra.run.trial195.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-20T17:56:50.639 INFO:teuthology.orchestra.run.trial195.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-20T17:56:50.639 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:50.640 INFO:teuthology.orchestra.run.trial195.stdout:Transaction Summary 2026-02-20T17:56:50.640 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:56:50.640 INFO:teuthology.orchestra.run.trial195.stdout:Install 10 Packages 2026-02-20T17:56:50.640 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:50.640 INFO:teuthology.orchestra.run.trial195.stdout:Total download size: 2.0 M 2026-02-20T17:56:50.640 INFO:teuthology.orchestra.run.trial195.stdout:Installed size: 25 M 2026-02-20T17:56:50.640 INFO:teuthology.orchestra.run.trial195.stdout:Downloading Packages: 2026-02-20T17:56:50.881 INFO:teuthology.orchestra.run.trial195.stdout:(1/10): ceph-mgr-rook-20.2.0-677.gf7870454.el9. 688 kB/s | 50 kB 00:00 2026-02-20T17:56:51.457 INFO:teuthology.orchestra.run.trial195.stdout:(2/10): python3-requests-oauthlib-1.3.0-12.el9. 93 kB/s | 54 kB 00:00 2026-02-20T17:56:51.532 INFO:teuthology.orchestra.run.trial195.stdout:(3/10): python3-pyasn1-0.4.8-6.el9.noarch.rpm 219 kB/s | 159 kB 00:00 2026-02-20T17:56:51.585 INFO:teuthology.orchestra.run.trial195.stdout:(4/10): python3-pyasn1-modules-0.4.8-6.el9.noar 358 kB/s | 279 kB 00:00 2026-02-20T17:56:51.637 INFO:teuthology.orchestra.run.trial195.stdout:(5/10): python3-cachetools-4.2.4-1.el9.noarch.r 179 kB/s | 32 kB 00:00 2026-02-20T17:56:51.657 INFO:teuthology.orchestra.run.trial195.stdout:(6/10): python3-certifi-2023.05.07-4.el9.noarch 113 kB/s | 14 kB 00:00 2026-02-20T17:56:51.750 INFO:teuthology.orchestra.run.trial195.stdout:(7/10): python3-rsa-4.9-2.el9.noarch.rpm 641 kB/s | 59 kB 00:00 2026-02-20T17:56:51.818 INFO:teuthology.orchestra.run.trial195.stdout:(8/10): python3-websocket-client-1.2.3-2.el9.no 1.3 MB/s | 90 kB 00:00 2026-02-20T17:56:51.862 INFO:teuthology.orchestra.run.trial195.stdout:(9/10): python3-google-auth-2.45.0-1.el9.noarch 921 kB/s | 254 kB 00:00 2026-02-20T17:56:51.866 INFO:teuthology.orchestra.run.trial195.stdout:(10/10): python3-kubernetes-26.1.0-3.el9.noarch 4.5 MB/s | 1.0 MB 00:00 2026-02-20T17:56:51.868 INFO:teuthology.orchestra.run.trial195.stdout:-------------------------------------------------------------------------------- 2026-02-20T17:56:51.868 INFO:teuthology.orchestra.run.trial195.stdout:Total 1.6 MB/s | 2.0 MB 00:01 2026-02-20T17:56:51.894 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction check 2026-02-20T17:56:51.901 INFO:teuthology.orchestra.run.trial195.stdout:Transaction check succeeded. 2026-02-20T17:56:51.901 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction test 2026-02-20T17:56:51.940 INFO:teuthology.orchestra.run.trial195.stdout:Transaction test succeeded. 2026-02-20T17:56:51.941 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction 2026-02-20T17:56:52.014 INFO:teuthology.orchestra.run.trial195.stdout: Preparing : 1/1 2026-02-20T17:56:52.065 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/10 2026-02-20T17:56:52.077 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/10 2026-02-20T17:56:52.092 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/10 2026-02-20T17:56:52.096 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/10 2026-02-20T17:56:52.104 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/10 2026-02-20T17:56:52.143 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/10 2026-02-20T17:56:52.155 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 7/10 2026-02-20T17:56:52.472 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 8/10 2026-02-20T17:56:52.486 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 9/10 2026-02-20T17:56:52.487 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-mgr-rook-2:20.2.0-677.gf7870454.el9.noarch 10/10 2026-02-20T17:56:52.594 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: ceph-mgr-rook-2:20.2.0-677.gf7870454.el9.noarch 10/10 2026-02-20T17:56:52.595 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-mgr-rook-2:20.2.0-677.gf7870454.el9.noarch 1/10 2026-02-20T17:56:52.595 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 2/10 2026-02-20T17:56:52.595 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 3/10 2026-02-20T17:56:52.595 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 4/10 2026-02-20T17:56:52.595 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 5/10 2026-02-20T17:56:52.595 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 6/10 2026-02-20T17:56:52.595 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 7/10 2026-02-20T17:56:52.595 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 8/10 2026-02-20T17:56:52.595 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 9/10 2026-02-20T17:56:52.651 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 10/10 2026-02-20T17:56:52.652 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:52.652 INFO:teuthology.orchestra.run.trial195.stdout:Installed: 2026-02-20T17:56:52.652 INFO:teuthology.orchestra.run.trial195.stdout: ceph-mgr-rook-2:20.2.0-677.gf7870454.el9.noarch 2026-02-20T17:56:52.652 INFO:teuthology.orchestra.run.trial195.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-20T17:56:52.652 INFO:teuthology.orchestra.run.trial195.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-20T17:56:52.652 INFO:teuthology.orchestra.run.trial195.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-20T17:56:52.652 INFO:teuthology.orchestra.run.trial195.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-20T17:56:52.652 INFO:teuthology.orchestra.run.trial195.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-20T17:56:52.652 INFO:teuthology.orchestra.run.trial195.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-20T17:56:52.652 INFO:teuthology.orchestra.run.trial195.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-20T17:56:52.652 INFO:teuthology.orchestra.run.trial195.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-20T17:56:52.652 INFO:teuthology.orchestra.run.trial195.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-20T17:56:52.652 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:52.652 INFO:teuthology.orchestra.run.trial195.stdout:Complete! 2026-02-20T17:56:52.684 DEBUG:teuthology.orchestra.run.trial195:> sudo yum -y install ceph-mgr-cephadm 2026-02-20T17:56:52.998 INFO:teuthology.orchestra.run.trial195.stdout:Last metadata expiration check: 0:00:38 ago on Fri 20 Feb 2026 05:56:14 PM UTC. 2026-02-20T17:56:53.082 INFO:teuthology.orchestra.run.trial195.stdout:Dependencies resolved. 2026-02-20T17:56:53.083 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:56:53.083 INFO:teuthology.orchestra.run.trial195.stdout: Package Arch Version Repository Size 2026-02-20T17:56:53.083 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:56:53.084 INFO:teuthology.orchestra.run.trial195.stdout:Installing: 2026-02-20T17:56:53.084 INFO:teuthology.orchestra.run.trial195.stdout: ceph-mgr-cephadm noarch 2:20.2.0-677.gf7870454.el9 ceph-noarch 173 k 2026-02-20T17:56:53.084 INFO:teuthology.orchestra.run.trial195.stdout:Installing dependencies: 2026-02-20T17:56:53.084 INFO:teuthology.orchestra.run.trial195.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-20T17:56:53.084 INFO:teuthology.orchestra.run.trial195.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-20T17:56:53.084 INFO:teuthology.orchestra.run.trial195.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-20T17:56:53.084 INFO:teuthology.orchestra.run.trial195.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-20T17:56:53.085 INFO:teuthology.orchestra.run.trial195.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-20T17:56:53.085 INFO:teuthology.orchestra.run.trial195.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-20T17:56:53.085 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:53.085 INFO:teuthology.orchestra.run.trial195.stdout:Transaction Summary 2026-02-20T17:56:53.085 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:56:53.085 INFO:teuthology.orchestra.run.trial195.stdout:Install 7 Packages 2026-02-20T17:56:53.085 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:53.085 INFO:teuthology.orchestra.run.trial195.stdout:Total download size: 7.1 M 2026-02-20T17:56:53.085 INFO:teuthology.orchestra.run.trial195.stdout:Installed size: 33 M 2026-02-20T17:56:53.085 INFO:teuthology.orchestra.run.trial195.stdout:Downloading Packages: 2026-02-20T17:56:53.506 INFO:teuthology.orchestra.run.trial195.stdout:(1/7): python3-jinja2-2.11.3-8.el9.noarch.rpm 11 MB/s | 249 kB 00:00 2026-02-20T17:56:53.510 INFO:teuthology.orchestra.run.trial195.stdout:(2/7): python3-markupsafe-1.1.1-12.el9.x86_64.r 7.0 MB/s | 35 kB 00:00 2026-02-20T17:56:53.527 INFO:teuthology.orchestra.run.trial195.stdout:(3/7): python3-babel-2.9.1-2.el9.noarch.rpm 136 MB/s | 6.0 MB 00:00 2026-02-20T17:56:53.572 INFO:teuthology.orchestra.run.trial195.stdout:(4/7): ceph-mgr-cephadm-20.2.0-677.gf7870454.el 1.9 MB/s | 173 kB 00:00 2026-02-20T17:56:55.132 INFO:teuthology.orchestra.run.trial195.stdout:(5/7): python3-typing-extensions-4.15.0-1.el9.n 55 kB/s | 86 kB 00:01 2026-02-20T17:56:55.148 INFO:teuthology.orchestra.run.trial195.stdout:(6/7): python3-natsort-7.1.1-5.el9.noarch.rpm 36 kB/s | 58 kB 00:01 2026-02-20T17:56:55.166 INFO:teuthology.orchestra.run.trial195.stdout:(7/7): python3-asyncssh-2.13.2-5.el9.noarch.rpm 331 kB/s | 548 kB 00:01 2026-02-20T17:56:55.167 INFO:teuthology.orchestra.run.trial195.stdout:-------------------------------------------------------------------------------- 2026-02-20T17:56:55.167 INFO:teuthology.orchestra.run.trial195.stdout:Total 3.4 MB/s | 7.1 MB 00:02 2026-02-20T17:56:55.204 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction check 2026-02-20T17:56:55.211 INFO:teuthology.orchestra.run.trial195.stdout:Transaction check succeeded. 2026-02-20T17:56:55.211 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction test 2026-02-20T17:56:55.281 INFO:teuthology.orchestra.run.trial195.stdout:Transaction test succeeded. 2026-02-20T17:56:55.281 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction 2026-02-20T17:56:55.371 INFO:teuthology.orchestra.run.trial195.stdout: Preparing : 1/1 2026-02-20T17:56:55.420 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 1/7 2026-02-20T17:56:55.430 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/7 2026-02-20T17:56:55.436 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/7 2026-02-20T17:56:55.656 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 4/7 2026-02-20T17:56:55.680 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 5/7 2026-02-20T17:56:55.703 INFO:teuthology.orchestra.run.trial195.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 6/7 2026-02-20T17:56:55.704 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-mgr-cephadm-2:20.2.0-677.gf7870454.el9.noarch 7/7 2026-02-20T17:56:55.812 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.2.0-677.gf7870454.el9.noarch 7/7 2026-02-20T17:56:55.813 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-mgr-cephadm-2:20.2.0-677.gf7870454.el9.noarch 1/7 2026-02-20T17:56:55.813 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/7 2026-02-20T17:56:55.813 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 3/7 2026-02-20T17:56:55.813 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 4/7 2026-02-20T17:56:55.813 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 5/7 2026-02-20T17:56:55.813 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 6/7 2026-02-20T17:56:55.870 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 7/7 2026-02-20T17:56:55.870 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:55.870 INFO:teuthology.orchestra.run.trial195.stdout:Installed: 2026-02-20T17:56:55.870 INFO:teuthology.orchestra.run.trial195.stdout: ceph-mgr-cephadm-2:20.2.0-677.gf7870454.el9.noarch 2026-02-20T17:56:55.870 INFO:teuthology.orchestra.run.trial195.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-20T17:56:55.870 INFO:teuthology.orchestra.run.trial195.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-20T17:56:55.870 INFO:teuthology.orchestra.run.trial195.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-20T17:56:55.870 INFO:teuthology.orchestra.run.trial195.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-20T17:56:55.870 INFO:teuthology.orchestra.run.trial195.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-20T17:56:55.870 INFO:teuthology.orchestra.run.trial195.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-20T17:56:55.870 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:55.870 INFO:teuthology.orchestra.run.trial195.stdout:Complete! 2026-02-20T17:56:55.902 DEBUG:teuthology.orchestra.run.trial195:> sudo yum -y install ceph-fuse 2026-02-20T17:56:56.200 INFO:teuthology.orchestra.run.trial195.stdout:Last metadata expiration check: 0:00:42 ago on Fri 20 Feb 2026 05:56:14 PM UTC. 2026-02-20T17:56:56.283 INFO:teuthology.orchestra.run.trial195.stdout:Dependencies resolved. 2026-02-20T17:56:56.284 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:56:56.284 INFO:teuthology.orchestra.run.trial195.stdout: Package Architecture Version Repository Size 2026-02-20T17:56:56.284 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:56:56.284 INFO:teuthology.orchestra.run.trial195.stdout:Installing: 2026-02-20T17:56:56.284 INFO:teuthology.orchestra.run.trial195.stdout: ceph-fuse x86_64 2:20.2.0-677.gf7870454.el9 ceph 943 k 2026-02-20T17:56:56.284 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:56.284 INFO:teuthology.orchestra.run.trial195.stdout:Transaction Summary 2026-02-20T17:56:56.284 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:56:56.284 INFO:teuthology.orchestra.run.trial195.stdout:Install 1 Package 2026-02-20T17:56:56.285 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:56.285 INFO:teuthology.orchestra.run.trial195.stdout:Total download size: 943 k 2026-02-20T17:56:56.285 INFO:teuthology.orchestra.run.trial195.stdout:Installed size: 2.7 M 2026-02-20T17:56:56.285 INFO:teuthology.orchestra.run.trial195.stdout:Downloading Packages: 2026-02-20T17:56:56.403 INFO:teuthology.orchestra.run.trial195.stdout:ceph-fuse-20.2.0-677.gf7870454.el9.x86_64.rpm 7.7 MB/s | 943 kB 00:00 2026-02-20T17:56:56.404 INFO:teuthology.orchestra.run.trial195.stdout:-------------------------------------------------------------------------------- 2026-02-20T17:56:56.404 INFO:teuthology.orchestra.run.trial195.stdout:Total 7.7 MB/s | 943 kB 00:00 2026-02-20T17:56:56.404 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction check 2026-02-20T17:56:56.408 INFO:teuthology.orchestra.run.trial195.stdout:Transaction check succeeded. 2026-02-20T17:56:56.408 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction test 2026-02-20T17:56:56.441 INFO:teuthology.orchestra.run.trial195.stdout:Transaction test succeeded. 2026-02-20T17:56:56.442 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction 2026-02-20T17:56:56.530 INFO:teuthology.orchestra.run.trial195.stdout: Preparing : 1/1 2026-02-20T17:56:56.546 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-fuse-2:20.2.0-677.gf7870454.el9.x86_64 1/1 2026-02-20T17:56:56.783 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: ceph-fuse-2:20.2.0-677.gf7870454.el9.x86_64 1/1 2026-02-20T17:56:56.839 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-fuse-2:20.2.0-677.gf7870454.el9.x86_64 1/1 2026-02-20T17:56:56.840 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:56.840 INFO:teuthology.orchestra.run.trial195.stdout:Installed: 2026-02-20T17:56:56.840 INFO:teuthology.orchestra.run.trial195.stdout: ceph-fuse-2:20.2.0-677.gf7870454.el9.x86_64 2026-02-20T17:56:56.840 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:56.840 INFO:teuthology.orchestra.run.trial195.stdout:Complete! 2026-02-20T17:56:56.867 DEBUG:teuthology.orchestra.run.trial195:> sudo yum -y install ceph-volume 2026-02-20T17:56:57.179 INFO:teuthology.orchestra.run.trial195.stdout:Last metadata expiration check: 0:00:43 ago on Fri 20 Feb 2026 05:56:14 PM UTC. 2026-02-20T17:56:57.264 INFO:teuthology.orchestra.run.trial195.stdout:Dependencies resolved. 2026-02-20T17:56:57.264 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:56:57.265 INFO:teuthology.orchestra.run.trial195.stdout: Package Arch Version Repository Size 2026-02-20T17:56:57.265 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:56:57.265 INFO:teuthology.orchestra.run.trial195.stdout:Installing: 2026-02-20T17:56:57.265 INFO:teuthology.orchestra.run.trial195.stdout: ceph-volume noarch 2:20.2.0-677.gf7870454.el9 ceph-noarch 297 k 2026-02-20T17:56:57.265 INFO:teuthology.orchestra.run.trial195.stdout:Installing dependencies: 2026-02-20T17:56:57.265 INFO:teuthology.orchestra.run.trial195.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-20T17:56:57.265 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:57.265 INFO:teuthology.orchestra.run.trial195.stdout:Transaction Summary 2026-02-20T17:56:57.265 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:56:57.265 INFO:teuthology.orchestra.run.trial195.stdout:Install 2 Packages 2026-02-20T17:56:57.265 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:57.265 INFO:teuthology.orchestra.run.trial195.stdout:Total download size: 648 k 2026-02-20T17:56:57.265 INFO:teuthology.orchestra.run.trial195.stdout:Installed size: 2.2 M 2026-02-20T17:56:57.265 INFO:teuthology.orchestra.run.trial195.stdout:Downloading Packages: 2026-02-20T17:56:57.423 INFO:teuthology.orchestra.run.trial195.stdout:(1/2): ceph-volume-20.2.0-677.gf7870454.el9.noa 3.1 MB/s | 297 kB 00:00 2026-02-20T17:56:58.132 INFO:teuthology.orchestra.run.trial195.stdout:(2/2): cryptsetup-2.8.1-3.el9.x86_64.rpm 436 kB/s | 351 kB 00:00 2026-02-20T17:56:58.132 INFO:teuthology.orchestra.run.trial195.stdout:-------------------------------------------------------------------------------- 2026-02-20T17:56:58.133 INFO:teuthology.orchestra.run.trial195.stdout:Total 747 kB/s | 648 kB 00:00 2026-02-20T17:56:58.141 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction check 2026-02-20T17:56:58.147 INFO:teuthology.orchestra.run.trial195.stdout:Transaction check succeeded. 2026-02-20T17:56:58.147 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction test 2026-02-20T17:56:58.190 INFO:teuthology.orchestra.run.trial195.stdout:Transaction test succeeded. 2026-02-20T17:56:58.190 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction 2026-02-20T17:56:58.269 INFO:teuthology.orchestra.run.trial195.stdout: Preparing : 1/1 2026-02-20T17:56:58.331 INFO:teuthology.orchestra.run.trial195.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 1/2 2026-02-20T17:56:58.333 INFO:teuthology.orchestra.run.trial195.stdout: Installing : ceph-volume-2:20.2.0-677.gf7870454.el9.noarch 2/2 2026-02-20T17:56:58.349 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: ceph-volume-2:20.2.0-677.gf7870454.el9.noarch 2/2 2026-02-20T17:56:58.349 INFO:teuthology.orchestra.run.trial195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T17:56:58.349 INFO:teuthology.orchestra.run.trial195.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-20T17:56:58.349 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:58.570 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : ceph-volume-2:20.2.0-677.gf7870454.el9.noarch 1/2 2026-02-20T17:56:58.629 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 2/2 2026-02-20T17:56:58.629 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:58.630 INFO:teuthology.orchestra.run.trial195.stdout:Installed: 2026-02-20T17:56:58.630 INFO:teuthology.orchestra.run.trial195.stdout: ceph-volume-2:20.2.0-677.gf7870454.el9.noarch cryptsetup-2.8.1-3.el9.x86_64 2026-02-20T17:56:58.630 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:58.630 INFO:teuthology.orchestra.run.trial195.stdout:Complete! 2026-02-20T17:56:58.658 DEBUG:teuthology.orchestra.run.trial195:> sudo yum -y install librados-devel 2026-02-20T17:56:58.973 INFO:teuthology.orchestra.run.trial195.stdout:Last metadata expiration check: 0:00:44 ago on Fri 20 Feb 2026 05:56:14 PM UTC. 2026-02-20T17:56:59.054 INFO:teuthology.orchestra.run.trial195.stdout:Dependencies resolved. 2026-02-20T17:56:59.055 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:56:59.055 INFO:teuthology.orchestra.run.trial195.stdout: Package Arch Version Repo Size 2026-02-20T17:56:59.056 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:56:59.056 INFO:teuthology.orchestra.run.trial195.stdout:Installing: 2026-02-20T17:56:59.056 INFO:teuthology.orchestra.run.trial195.stdout: librados-devel x86_64 2:20.2.0-677.gf7870454.el9 ceph 126 k 2026-02-20T17:56:59.056 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:59.056 INFO:teuthology.orchestra.run.trial195.stdout:Transaction Summary 2026-02-20T17:56:59.056 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:56:59.056 INFO:teuthology.orchestra.run.trial195.stdout:Install 1 Package 2026-02-20T17:56:59.056 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:59.056 INFO:teuthology.orchestra.run.trial195.stdout:Total download size: 126 k 2026-02-20T17:56:59.056 INFO:teuthology.orchestra.run.trial195.stdout:Installed size: 449 k 2026-02-20T17:56:59.056 INFO:teuthology.orchestra.run.trial195.stdout:Downloading Packages: 2026-02-20T17:56:59.140 INFO:teuthology.orchestra.run.trial195.stdout:librados-devel-20.2.0-677.gf7870454.el9.x86_64. 1.5 MB/s | 126 kB 00:00 2026-02-20T17:56:59.140 INFO:teuthology.orchestra.run.trial195.stdout:-------------------------------------------------------------------------------- 2026-02-20T17:56:59.141 INFO:teuthology.orchestra.run.trial195.stdout:Total 1.4 MB/s | 126 kB 00:00 2026-02-20T17:56:59.141 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction check 2026-02-20T17:56:59.143 INFO:teuthology.orchestra.run.trial195.stdout:Transaction check succeeded. 2026-02-20T17:56:59.143 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction test 2026-02-20T17:56:59.173 INFO:teuthology.orchestra.run.trial195.stdout:Transaction test succeeded. 2026-02-20T17:56:59.174 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction 2026-02-20T17:56:59.228 INFO:teuthology.orchestra.run.trial195.stdout: Preparing : 1/1 2026-02-20T17:56:59.244 INFO:teuthology.orchestra.run.trial195.stdout: Installing : librados-devel-2:20.2.0-677.gf7870454.el9.x86_64 1/1 2026-02-20T17:56:59.324 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: librados-devel-2:20.2.0-677.gf7870454.el9.x86_64 1/1 2026-02-20T17:56:59.381 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : librados-devel-2:20.2.0-677.gf7870454.el9.x86_64 1/1 2026-02-20T17:56:59.381 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:59.381 INFO:teuthology.orchestra.run.trial195.stdout:Installed: 2026-02-20T17:56:59.382 INFO:teuthology.orchestra.run.trial195.stdout: librados-devel-2:20.2.0-677.gf7870454.el9.x86_64 2026-02-20T17:56:59.382 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:56:59.382 INFO:teuthology.orchestra.run.trial195.stdout:Complete! 2026-02-20T17:56:59.408 DEBUG:teuthology.orchestra.run.trial195:> sudo yum -y install libcephfs2 2026-02-20T17:56:59.723 INFO:teuthology.orchestra.run.trial195.stdout:Last metadata expiration check: 0:00:45 ago on Fri 20 Feb 2026 05:56:14 PM UTC. 2026-02-20T17:56:59.789 INFO:teuthology.orchestra.run.trial195.stdout:Package libcephfs2-2:20.2.0-677.gf7870454.el9.x86_64 is already installed. 2026-02-20T17:56:59.807 INFO:teuthology.orchestra.run.trial195.stdout:Dependencies resolved. 2026-02-20T17:56:59.808 INFO:teuthology.orchestra.run.trial195.stdout:Nothing to do. 2026-02-20T17:56:59.808 INFO:teuthology.orchestra.run.trial195.stdout:Complete! 2026-02-20T17:56:59.829 DEBUG:teuthology.orchestra.run.trial195:> sudo yum -y install libcephfs-devel 2026-02-20T17:57:00.145 INFO:teuthology.orchestra.run.trial195.stdout:Last metadata expiration check: 0:00:46 ago on Fri 20 Feb 2026 05:56:14 PM UTC. 2026-02-20T17:57:00.227 INFO:teuthology.orchestra.run.trial195.stdout:Dependencies resolved. 2026-02-20T17:57:00.228 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:57:00.228 INFO:teuthology.orchestra.run.trial195.stdout: Package Arch Version Repo Size 2026-02-20T17:57:00.228 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:57:00.228 INFO:teuthology.orchestra.run.trial195.stdout:Installing: 2026-02-20T17:57:00.228 INFO:teuthology.orchestra.run.trial195.stdout: libcephfs-devel x86_64 2:20.2.0-677.gf7870454.el9 ceph 34 k 2026-02-20T17:57:00.228 INFO:teuthology.orchestra.run.trial195.stdout:Installing dependencies: 2026-02-20T17:57:00.228 INFO:teuthology.orchestra.run.trial195.stdout: libcephfs-proxy2 x86_64 2:20.2.0-677.gf7870454.el9 ceph 24 k 2026-02-20T17:57:00.228 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:57:00.229 INFO:teuthology.orchestra.run.trial195.stdout:Transaction Summary 2026-02-20T17:57:00.229 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:57:00.229 INFO:teuthology.orchestra.run.trial195.stdout:Install 2 Packages 2026-02-20T17:57:00.229 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:57:00.229 INFO:teuthology.orchestra.run.trial195.stdout:Total download size: 58 k 2026-02-20T17:57:00.229 INFO:teuthology.orchestra.run.trial195.stdout:Installed size: 207 k 2026-02-20T17:57:00.229 INFO:teuthology.orchestra.run.trial195.stdout:Downloading Packages: 2026-02-20T17:57:00.292 INFO:teuthology.orchestra.run.trial195.stdout:(1/2): libcephfs-proxy2-20.2.0-677.gf7870454.el 378 kB/s | 24 kB 00:00 2026-02-20T17:57:00.293 INFO:teuthology.orchestra.run.trial195.stdout:(2/2): libcephfs-devel-20.2.0-677.gf7870454.el9 526 kB/s | 34 kB 00:00 2026-02-20T17:57:00.294 INFO:teuthology.orchestra.run.trial195.stdout:-------------------------------------------------------------------------------- 2026-02-20T17:57:00.294 INFO:teuthology.orchestra.run.trial195.stdout:Total 876 kB/s | 58 kB 00:00 2026-02-20T17:57:00.295 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction check 2026-02-20T17:57:00.297 INFO:teuthology.orchestra.run.trial195.stdout:Transaction check succeeded. 2026-02-20T17:57:00.298 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction test 2026-02-20T17:57:00.329 INFO:teuthology.orchestra.run.trial195.stdout:Transaction test succeeded. 2026-02-20T17:57:00.329 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction 2026-02-20T17:57:00.380 INFO:teuthology.orchestra.run.trial195.stdout: Preparing : 1/1 2026-02-20T17:57:00.381 INFO:teuthology.orchestra.run.trial195.stdout: Installing : libcephfs-proxy2-2:20.2.0-677.gf7870454.el9.x86_64 1/2 2026-02-20T17:57:00.397 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: libcephfs-proxy2-2:20.2.0-677.gf7870454.el9.x86_64 1/2 2026-02-20T17:57:00.414 INFO:teuthology.orchestra.run.trial195.stdout: Installing : libcephfs-devel-2:20.2.0-677.gf7870454.el9.x86_64 2/2 2026-02-20T17:57:00.466 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: libcephfs-devel-2:20.2.0-677.gf7870454.el9.x86_64 2/2 2026-02-20T17:57:00.467 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : libcephfs-devel-2:20.2.0-677.gf7870454.el9.x86_64 1/2 2026-02-20T17:57:00.523 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : libcephfs-proxy2-2:20.2.0-677.gf7870454.el9.x86_64 2/2 2026-02-20T17:57:00.523 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:57:00.523 INFO:teuthology.orchestra.run.trial195.stdout:Installed: 2026-02-20T17:57:00.523 INFO:teuthology.orchestra.run.trial195.stdout: libcephfs-devel-2:20.2.0-677.gf7870454.el9.x86_64 2026-02-20T17:57:00.523 INFO:teuthology.orchestra.run.trial195.stdout: libcephfs-proxy2-2:20.2.0-677.gf7870454.el9.x86_64 2026-02-20T17:57:00.523 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:57:00.523 INFO:teuthology.orchestra.run.trial195.stdout:Complete! 2026-02-20T17:57:00.549 DEBUG:teuthology.orchestra.run.trial195:> sudo yum -y install librados2 2026-02-20T17:57:00.864 INFO:teuthology.orchestra.run.trial195.stdout:Last metadata expiration check: 0:00:46 ago on Fri 20 Feb 2026 05:56:14 PM UTC. 2026-02-20T17:57:00.930 INFO:teuthology.orchestra.run.trial195.stdout:Package librados2-2:20.2.0-677.gf7870454.el9.x86_64 is already installed. 2026-02-20T17:57:00.948 INFO:teuthology.orchestra.run.trial195.stdout:Dependencies resolved. 2026-02-20T17:57:00.949 INFO:teuthology.orchestra.run.trial195.stdout:Nothing to do. 2026-02-20T17:57:00.949 INFO:teuthology.orchestra.run.trial195.stdout:Complete! 2026-02-20T17:57:00.971 DEBUG:teuthology.orchestra.run.trial195:> sudo yum -y install librbd1 2026-02-20T17:57:01.282 INFO:teuthology.orchestra.run.trial195.stdout:Last metadata expiration check: 0:00:47 ago on Fri 20 Feb 2026 05:56:14 PM UTC. 2026-02-20T17:57:01.346 INFO:teuthology.orchestra.run.trial195.stdout:Package librbd1-2:20.2.0-677.gf7870454.el9.x86_64 is already installed. 2026-02-20T17:57:01.364 INFO:teuthology.orchestra.run.trial195.stdout:Dependencies resolved. 2026-02-20T17:57:01.365 INFO:teuthology.orchestra.run.trial195.stdout:Nothing to do. 2026-02-20T17:57:01.365 INFO:teuthology.orchestra.run.trial195.stdout:Complete! 2026-02-20T17:57:01.386 DEBUG:teuthology.orchestra.run.trial195:> sudo yum -y install python3-rados 2026-02-20T17:57:01.694 INFO:teuthology.orchestra.run.trial195.stdout:Last metadata expiration check: 0:00:47 ago on Fri 20 Feb 2026 05:56:14 PM UTC. 2026-02-20T17:57:01.758 INFO:teuthology.orchestra.run.trial195.stdout:Package python3-rados-2:20.2.0-677.gf7870454.el9.x86_64 is already installed. 2026-02-20T17:57:01.776 INFO:teuthology.orchestra.run.trial195.stdout:Dependencies resolved. 2026-02-20T17:57:01.777 INFO:teuthology.orchestra.run.trial195.stdout:Nothing to do. 2026-02-20T17:57:01.777 INFO:teuthology.orchestra.run.trial195.stdout:Complete! 2026-02-20T17:57:01.799 DEBUG:teuthology.orchestra.run.trial195:> sudo yum -y install python3-rgw 2026-02-20T17:57:02.111 INFO:teuthology.orchestra.run.trial195.stdout:Last metadata expiration check: 0:00:48 ago on Fri 20 Feb 2026 05:56:14 PM UTC. 2026-02-20T17:57:02.176 INFO:teuthology.orchestra.run.trial195.stdout:Package python3-rgw-2:20.2.0-677.gf7870454.el9.x86_64 is already installed. 2026-02-20T17:57:02.195 INFO:teuthology.orchestra.run.trial195.stdout:Dependencies resolved. 2026-02-20T17:57:02.196 INFO:teuthology.orchestra.run.trial195.stdout:Nothing to do. 2026-02-20T17:57:02.196 INFO:teuthology.orchestra.run.trial195.stdout:Complete! 2026-02-20T17:57:02.217 DEBUG:teuthology.orchestra.run.trial195:> sudo yum -y install python3-cephfs 2026-02-20T17:57:02.532 INFO:teuthology.orchestra.run.trial195.stdout:Last metadata expiration check: 0:00:48 ago on Fri 20 Feb 2026 05:56:14 PM UTC. 2026-02-20T17:57:02.596 INFO:teuthology.orchestra.run.trial195.stdout:Package python3-cephfs-2:20.2.0-677.gf7870454.el9.x86_64 is already installed. 2026-02-20T17:57:02.614 INFO:teuthology.orchestra.run.trial195.stdout:Dependencies resolved. 2026-02-20T17:57:02.615 INFO:teuthology.orchestra.run.trial195.stdout:Nothing to do. 2026-02-20T17:57:02.615 INFO:teuthology.orchestra.run.trial195.stdout:Complete! 2026-02-20T17:57:02.637 DEBUG:teuthology.orchestra.run.trial195:> sudo yum -y install python3-rbd 2026-02-20T17:57:02.947 INFO:teuthology.orchestra.run.trial195.stdout:Last metadata expiration check: 0:00:48 ago on Fri 20 Feb 2026 05:56:14 PM UTC. 2026-02-20T17:57:03.014 INFO:teuthology.orchestra.run.trial195.stdout:Package python3-rbd-2:20.2.0-677.gf7870454.el9.x86_64 is already installed. 2026-02-20T17:57:03.033 INFO:teuthology.orchestra.run.trial195.stdout:Dependencies resolved. 2026-02-20T17:57:03.033 INFO:teuthology.orchestra.run.trial195.stdout:Nothing to do. 2026-02-20T17:57:03.033 INFO:teuthology.orchestra.run.trial195.stdout:Complete! 2026-02-20T17:57:03.055 DEBUG:teuthology.orchestra.run.trial195:> sudo yum -y install rbd-fuse 2026-02-20T17:57:03.360 INFO:teuthology.orchestra.run.trial195.stdout:Last metadata expiration check: 0:00:49 ago on Fri 20 Feb 2026 05:56:14 PM UTC. 2026-02-20T17:57:03.445 INFO:teuthology.orchestra.run.trial195.stdout:Dependencies resolved. 2026-02-20T17:57:03.446 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:57:03.446 INFO:teuthology.orchestra.run.trial195.stdout: Package Architecture Version Repository Size 2026-02-20T17:57:03.446 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:57:03.446 INFO:teuthology.orchestra.run.trial195.stdout:Installing: 2026-02-20T17:57:03.446 INFO:teuthology.orchestra.run.trial195.stdout: rbd-fuse x86_64 2:20.2.0-677.gf7870454.el9 ceph 91 k 2026-02-20T17:57:03.446 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:57:03.446 INFO:teuthology.orchestra.run.trial195.stdout:Transaction Summary 2026-02-20T17:57:03.446 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:57:03.447 INFO:teuthology.orchestra.run.trial195.stdout:Install 1 Package 2026-02-20T17:57:03.447 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:57:03.447 INFO:teuthology.orchestra.run.trial195.stdout:Total download size: 91 k 2026-02-20T17:57:03.447 INFO:teuthology.orchestra.run.trial195.stdout:Installed size: 238 k 2026-02-20T17:57:03.447 INFO:teuthology.orchestra.run.trial195.stdout:Downloading Packages: 2026-02-20T17:57:03.521 INFO:teuthology.orchestra.run.trial195.stdout:rbd-fuse-20.2.0-677.gf7870454.el9.x86_64.rpm 1.2 MB/s | 91 kB 00:00 2026-02-20T17:57:03.521 INFO:teuthology.orchestra.run.trial195.stdout:-------------------------------------------------------------------------------- 2026-02-20T17:57:03.521 INFO:teuthology.orchestra.run.trial195.stdout:Total 1.2 MB/s | 91 kB 00:00 2026-02-20T17:57:03.521 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction check 2026-02-20T17:57:03.525 INFO:teuthology.orchestra.run.trial195.stdout:Transaction check succeeded. 2026-02-20T17:57:03.525 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction test 2026-02-20T17:57:03.556 INFO:teuthology.orchestra.run.trial195.stdout:Transaction test succeeded. 2026-02-20T17:57:03.557 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction 2026-02-20T17:57:03.608 INFO:teuthology.orchestra.run.trial195.stdout: Preparing : 1/1 2026-02-20T17:57:03.624 INFO:teuthology.orchestra.run.trial195.stdout: Installing : rbd-fuse-2:20.2.0-677.gf7870454.el9.x86_64 1/1 2026-02-20T17:57:03.702 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: rbd-fuse-2:20.2.0-677.gf7870454.el9.x86_64 1/1 2026-02-20T17:57:03.758 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : rbd-fuse-2:20.2.0-677.gf7870454.el9.x86_64 1/1 2026-02-20T17:57:03.758 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:57:03.758 INFO:teuthology.orchestra.run.trial195.stdout:Installed: 2026-02-20T17:57:03.758 INFO:teuthology.orchestra.run.trial195.stdout: rbd-fuse-2:20.2.0-677.gf7870454.el9.x86_64 2026-02-20T17:57:03.759 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:57:03.759 INFO:teuthology.orchestra.run.trial195.stdout:Complete! 2026-02-20T17:57:03.786 DEBUG:teuthology.orchestra.run.trial195:> sudo yum -y install rbd-mirror 2026-02-20T17:57:04.094 INFO:teuthology.orchestra.run.trial195.stdout:Last metadata expiration check: 0:00:50 ago on Fri 20 Feb 2026 05:56:14 PM UTC. 2026-02-20T17:57:04.178 INFO:teuthology.orchestra.run.trial195.stdout:Dependencies resolved. 2026-02-20T17:57:04.179 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:57:04.179 INFO:teuthology.orchestra.run.trial195.stdout: Package Arch Version Repo Size 2026-02-20T17:57:04.179 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:57:04.179 INFO:teuthology.orchestra.run.trial195.stdout:Installing: 2026-02-20T17:57:04.179 INFO:teuthology.orchestra.run.trial195.stdout: rbd-mirror x86_64 2:20.2.0-677.gf7870454.el9 ceph 2.9 M 2026-02-20T17:57:04.179 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:57:04.179 INFO:teuthology.orchestra.run.trial195.stdout:Transaction Summary 2026-02-20T17:57:04.179 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:57:04.179 INFO:teuthology.orchestra.run.trial195.stdout:Install 1 Package 2026-02-20T17:57:04.180 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:57:04.180 INFO:teuthology.orchestra.run.trial195.stdout:Total download size: 2.9 M 2026-02-20T17:57:04.180 INFO:teuthology.orchestra.run.trial195.stdout:Installed size: 11 M 2026-02-20T17:57:04.180 INFO:teuthology.orchestra.run.trial195.stdout:Downloading Packages: 2026-02-20T17:57:04.334 INFO:teuthology.orchestra.run.trial195.stdout:rbd-mirror-20.2.0-677.gf7870454.el9.x86_64.rpm 19 MB/s | 2.9 MB 00:00 2026-02-20T17:57:04.336 INFO:teuthology.orchestra.run.trial195.stdout:-------------------------------------------------------------------------------- 2026-02-20T17:57:04.336 INFO:teuthology.orchestra.run.trial195.stdout:Total 19 MB/s | 2.9 MB 00:00 2026-02-20T17:57:04.336 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction check 2026-02-20T17:57:04.340 INFO:teuthology.orchestra.run.trial195.stdout:Transaction check succeeded. 2026-02-20T17:57:04.341 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction test 2026-02-20T17:57:04.379 INFO:teuthology.orchestra.run.trial195.stdout:Transaction test succeeded. 2026-02-20T17:57:04.380 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction 2026-02-20T17:57:04.559 INFO:teuthology.orchestra.run.trial195.stdout: Preparing : 1/1 2026-02-20T17:57:04.561 INFO:teuthology.orchestra.run.trial195.stdout: Installing : rbd-mirror-2:20.2.0-677.gf7870454.el9.x86_64 1/1 2026-02-20T17:57:04.584 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: rbd-mirror-2:20.2.0-677.gf7870454.el9.x86_64 1/1 2026-02-20T17:57:04.584 INFO:teuthology.orchestra.run.trial195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T17:57:04.584 INFO:teuthology.orchestra.run.trial195.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-20T17:57:04.585 INFO:teuthology.orchestra.run.trial195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-20T17:57:04.585 INFO:teuthology.orchestra.run.trial195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-20T17:57:04.585 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:57:04.900 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : rbd-mirror-2:20.2.0-677.gf7870454.el9.x86_64 1/1 2026-02-20T17:57:04.900 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:57:04.900 INFO:teuthology.orchestra.run.trial195.stdout:Installed: 2026-02-20T17:57:04.900 INFO:teuthology.orchestra.run.trial195.stdout: rbd-mirror-2:20.2.0-677.gf7870454.el9.x86_64 2026-02-20T17:57:04.900 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:57:04.901 INFO:teuthology.orchestra.run.trial195.stdout:Complete! 2026-02-20T17:57:04.933 DEBUG:teuthology.orchestra.run.trial195:> sudo yum -y install rbd-nbd 2026-02-20T17:57:05.249 INFO:teuthology.orchestra.run.trial195.stdout:Last metadata expiration check: 0:00:51 ago on Fri 20 Feb 2026 05:56:14 PM UTC. 2026-02-20T17:57:05.333 INFO:teuthology.orchestra.run.trial195.stdout:Dependencies resolved. 2026-02-20T17:57:05.333 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:57:05.333 INFO:teuthology.orchestra.run.trial195.stdout: Package Architecture Version Repository Size 2026-02-20T17:57:05.333 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:57:05.333 INFO:teuthology.orchestra.run.trial195.stdout:Installing: 2026-02-20T17:57:05.333 INFO:teuthology.orchestra.run.trial195.stdout: rbd-nbd x86_64 2:20.2.0-677.gf7870454.el9 ceph 180 k 2026-02-20T17:57:05.333 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:57:05.334 INFO:teuthology.orchestra.run.trial195.stdout:Transaction Summary 2026-02-20T17:57:05.334 INFO:teuthology.orchestra.run.trial195.stdout:================================================================================ 2026-02-20T17:57:05.334 INFO:teuthology.orchestra.run.trial195.stdout:Install 1 Package 2026-02-20T17:57:05.334 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:57:05.334 INFO:teuthology.orchestra.run.trial195.stdout:Total download size: 180 k 2026-02-20T17:57:05.334 INFO:teuthology.orchestra.run.trial195.stdout:Installed size: 498 k 2026-02-20T17:57:05.334 INFO:teuthology.orchestra.run.trial195.stdout:Downloading Packages: 2026-02-20T17:57:05.418 INFO:teuthology.orchestra.run.trial195.stdout:rbd-nbd-20.2.0-677.gf7870454.el9.x86_64.rpm 2.1 MB/s | 180 kB 00:00 2026-02-20T17:57:05.418 INFO:teuthology.orchestra.run.trial195.stdout:-------------------------------------------------------------------------------- 2026-02-20T17:57:05.419 INFO:teuthology.orchestra.run.trial195.stdout:Total 2.1 MB/s | 180 kB 00:00 2026-02-20T17:57:05.419 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction check 2026-02-20T17:57:05.423 INFO:teuthology.orchestra.run.trial195.stdout:Transaction check succeeded. 2026-02-20T17:57:05.423 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction test 2026-02-20T17:57:05.454 INFO:teuthology.orchestra.run.trial195.stdout:Transaction test succeeded. 2026-02-20T17:57:05.454 INFO:teuthology.orchestra.run.trial195.stdout:Running transaction 2026-02-20T17:57:05.510 INFO:teuthology.orchestra.run.trial195.stdout: Preparing : 1/1 2026-02-20T17:57:05.526 INFO:teuthology.orchestra.run.trial195.stdout: Installing : rbd-nbd-2:20.2.0-677.gf7870454.el9.x86_64 1/1 2026-02-20T17:57:05.609 INFO:teuthology.orchestra.run.trial195.stdout: Running scriptlet: rbd-nbd-2:20.2.0-677.gf7870454.el9.x86_64 1/1 2026-02-20T17:57:05.667 INFO:teuthology.orchestra.run.trial195.stdout: Verifying : rbd-nbd-2:20.2.0-677.gf7870454.el9.x86_64 1/1 2026-02-20T17:57:05.667 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:57:05.667 INFO:teuthology.orchestra.run.trial195.stdout:Installed: 2026-02-20T17:57:05.668 INFO:teuthology.orchestra.run.trial195.stdout: rbd-nbd-2:20.2.0-677.gf7870454.el9.x86_64 2026-02-20T17:57:05.668 INFO:teuthology.orchestra.run.trial195.stdout: 2026-02-20T17:57:05.668 INFO:teuthology.orchestra.run.trial195.stdout:Complete! 2026-02-20T17:57:05.698 DEBUG:teuthology.parallel:result is None 2026-02-20T17:57:05.698 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f7870454149f22574086dd2ff55d050585136531 2026-02-20T17:57:05.816 DEBUG:teuthology.orchestra.run.trial195:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-20T17:57:05.837 INFO:teuthology.orchestra.run.trial195.stdout:20.2.0-677.gf7870454.el9 2026-02-20T17:57:05.838 INFO:teuthology.packaging:The installed version of ceph is 20.2.0-677.gf7870454.el9 2026-02-20T17:57:05.839 INFO:teuthology.task.install:The correct ceph version 20.2.0-677.gf7870454 is installed. 2026-02-20T17:57:05.841 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-20T17:57:05.842 DEBUG:teuthology.orchestra.run.trial195:> set -ex 2026-02-20T17:57:05.842 DEBUG:teuthology.orchestra.run.trial195:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-20T17:57:05.902 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-20T17:57:05.902 DEBUG:teuthology.orchestra.run.trial195:> set -ex 2026-02-20T17:57:05.902 DEBUG:teuthology.orchestra.run.trial195:> sudo dd of=/usr/bin/daemon-helper 2026-02-20T17:57:05.966 DEBUG:teuthology.orchestra.run.trial195:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-20T17:57:06.031 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-20T17:57:06.031 DEBUG:teuthology.orchestra.run.trial195:> set -ex 2026-02-20T17:57:06.031 DEBUG:teuthology.orchestra.run.trial195:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-20T17:57:06.102 DEBUG:teuthology.orchestra.run.trial195:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-20T17:57:06.166 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-20T17:57:06.166 DEBUG:teuthology.orchestra.run.trial195:> set -ex 2026-02-20T17:57:06.166 DEBUG:teuthology.orchestra.run.trial195:> sudo dd of=/usr/bin/stdin-killer 2026-02-20T17:57:06.230 DEBUG:teuthology.orchestra.run.trial195:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-20T17:57:06.295 INFO:teuthology.run_tasks:Running task workunit... 2026-02-20T17:57:06.320 INFO:tasks.workunit:Pulling workunits from ref f7870454149f22574086dd2ff55d050585136531 2026-02-20T17:57:06.321 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-20T17:57:06.321 INFO:tasks.workunit:timeout=3h 2026-02-20T17:57:06.321 INFO:tasks.workunit:cleanup=True 2026-02-20T17:57:06.321 DEBUG:teuthology.orchestra.run.trial195:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-20T17:57:06.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-20T17:57:06.352 INFO:teuthology.orchestra.run.trial195.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-20T17:57:06.352 DEBUG:teuthology.orchestra.run.trial195:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-20T17:57:06.407 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-20T17:57:06.408 DEBUG:teuthology.orchestra.run.trial195:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-20T17:57:06.465 DEBUG:teuthology.orchestra.run.trial195:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout f7870454149f22574086dd2ff55d050585136531 2026-02-20T17:57:06.527 INFO:tasks.workunit.client.0.trial195.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-20T17:57:26.010 INFO:tasks.workunit.client.0.trial195.stderr:Note: switching to 'f7870454149f22574086dd2ff55d050585136531'. 2026-02-20T17:57:26.010 INFO:tasks.workunit.client.0.trial195.stderr: 2026-02-20T17:57:26.011 INFO:tasks.workunit.client.0.trial195.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-20T17:57:26.011 INFO:tasks.workunit.client.0.trial195.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-20T17:57:26.011 INFO:tasks.workunit.client.0.trial195.stderr:state without impacting any branches by switching back to a branch. 2026-02-20T17:57:26.011 INFO:tasks.workunit.client.0.trial195.stderr: 2026-02-20T17:57:26.011 INFO:tasks.workunit.client.0.trial195.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-20T17:57:26.011 INFO:tasks.workunit.client.0.trial195.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-20T17:57:26.011 INFO:tasks.workunit.client.0.trial195.stderr: 2026-02-20T17:57:26.011 INFO:tasks.workunit.client.0.trial195.stderr: git switch -c 2026-02-20T17:57:26.011 INFO:tasks.workunit.client.0.trial195.stderr: 2026-02-20T17:57:26.011 INFO:tasks.workunit.client.0.trial195.stderr:Or undo this operation with: 2026-02-20T17:57:26.011 INFO:tasks.workunit.client.0.trial195.stderr: 2026-02-20T17:57:26.012 INFO:tasks.workunit.client.0.trial195.stderr: git switch - 2026-02-20T17:57:26.012 INFO:tasks.workunit.client.0.trial195.stderr: 2026-02-20T17:57:26.012 INFO:tasks.workunit.client.0.trial195.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-20T17:57:26.012 INFO:tasks.workunit.client.0.trial195.stderr: 2026-02-20T17:57:26.012 INFO:tasks.workunit.client.0.trial195.stderr:HEAD is now at f7870454149 ceph-dev-pipeline: configure 2026-02-20T17:57:26.017 DEBUG:teuthology.orchestra.run.trial195:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/standalone && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-20T17:57:26.078 DEBUG:teuthology.orchestra.run.trial195:> set -ex 2026-02-20T17:57:26.078 DEBUG:teuthology.orchestra.run.trial195:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-20T17:57:26.137 INFO:tasks.workunit:Running workunits matching erasure-code on client.0... 2026-02-20T17:57:26.138 INFO:tasks.workunit:Running workunit erasure-code/test-erasure-code-plugins.sh... 2026-02-20T17:57:26.138 DEBUG:teuthology.orchestra.run.trial195:workunit test erasure-code/test-erasure-code-plugins.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f7870454149f22574086dd2ff55d050585136531 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh 2026-02-20T17:57:26.198 INFO:tasks.workunit.client.0.trial195.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh 2026-02-20T17:57:26.199 INFO:tasks.workunit.client.0.trial195.stderr:++ TIMEOUT=300 2026-02-20T17:57:26.199 INFO:tasks.workunit.client.0.trial195.stderr:++ WAIT_FOR_CLEAN_TIMEOUT=90 2026-02-20T17:57:26.199 INFO:tasks.workunit.client.0.trial195.stderr:++ MAX_TIMEOUT=15 2026-02-20T17:57:26.199 INFO:tasks.workunit.client.0.trial195.stderr:++ PG_NUM=4 2026-02-20T17:57:26.199 INFO:tasks.workunit.client.0.trial195.stderr:++ TMPDIR=/tmp 2026-02-20T17:57:26.199 INFO:tasks.workunit.client.0.trial195.stderr:++ CEPH_BUILD_VIRTUALENV=/tmp 2026-02-20T17:57:26.199 INFO:tasks.workunit.client.0.trial195.stderr:++ TESTDIR=/home/ubuntu/cephtest 2026-02-20T17:57:26.199 INFO:tasks.workunit.client.0.trial195.stderr:+++ uname 2026-02-20T17:57:26.200 INFO:tasks.workunit.client.0.trial195.stderr:++ '[' Linux = FreeBSD ']' 2026-02-20T17:57:26.200 INFO:tasks.workunit.client.0.trial195.stderr:++ SED=sed 2026-02-20T17:57:26.201 INFO:tasks.workunit.client.0.trial195.stderr:++ AWK=awk 2026-02-20T17:57:26.201 INFO:tasks.workunit.client.0.trial195.stderr:+++ stty -a 2026-02-20T17:57:26.201 INFO:tasks.workunit.client.0.trial195.stderr:+++ head -1 2026-02-20T17:57:26.201 INFO:tasks.workunit.client.0.trial195.stderr:+++ sed -e 's/.*columns \([0-9]*\).*/\1/' 2026-02-20T17:57:26.201 INFO:tasks.workunit.client.0.trial195.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-20T17:57:26.202 INFO:tasks.workunit.client.0.trial195.stderr:++ termwidth= 2026-02-20T17:57:26.202 INFO:tasks.workunit.client.0.trial195.stderr:++ '[' -n '' -a '' '!=' 0 ']' 2026-02-20T17:57:26.202 INFO:tasks.workunit.client.0.trial195.stderr:++ DIFFCOLOPTS='-y ' 2026-02-20T17:57:26.202 INFO:tasks.workunit.client.0.trial195.stderr:++ KERNCORE=kernel.core_pattern 2026-02-20T17:57:26.202 INFO:tasks.workunit.client.0.trial195.stderr:++ EXTRA_OPTS= 2026-02-20T17:57:26.203 INFO:tasks.workunit.client.0.trial195.stderr:++ test '' = TESTS 2026-02-20T17:57:26.203 INFO:tasks.workunit.client.0.trial195.stderr:++ uname -m 2026-02-20T17:57:26.206 INFO:tasks.workunit.client.0.trial195.stderr:+ arch=x86_64 2026-02-20T17:57:26.206 INFO:tasks.workunit.client.0.trial195.stderr:+ case $arch in 2026-02-20T17:57:26.206 INFO:tasks.workunit.client.0.trial195.stderr:+ legacy_jerasure_plugins=(jerasure_generic jerasure_sse3 jerasure_sse4) 2026-02-20T17:57:26.206 INFO:tasks.workunit.client.0.trial195.stderr:+ legacy_shec_plugins=(shec_generic shec_sse3 shec_sse4) 2026-02-20T17:57:26.206 INFO:tasks.workunit.client.0.trial195.stderr:+ plugins=(jerasure shec lrc isa) 2026-02-20T17:57:26.206 INFO:tasks.workunit.client.0.trial195.stderr:+ main test-erasure-code-plugins 2026-02-20T17:57:26.207 INFO:tasks.workunit.client.0.trial195.stderr:+ local dir=td/test-erasure-code-plugins 2026-02-20T17:57:26.207 INFO:tasks.workunit.client.0.trial195.stderr:+ shift 2026-02-20T17:57:26.207 INFO:tasks.workunit.client.0.trial195.stderr:+ shopt -s -o xtrace 2026-02-20T17:57:26.207 INFO:tasks.workunit.client.0.trial195.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-20T17:57:26.207 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-20T17:57:26.207 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-20T17:57:26.207 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export PYTHONWARNINGS=ignore 2026-02-20T17:57:26.207 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: PYTHONWARNINGS=ignore 2026-02-20T17:57:26.207 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: export CEPH_CONF=/dev/null 2026-02-20T17:57:26.207 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: CEPH_CONF=/dev/null 2026-02-20T17:57:26.207 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2397: main: unset CEPH_ARGS 2026-02-20T17:57:26.207 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: local code 2026-02-20T17:57:26.208 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: run td/test-erasure-code-plugins 2026-02-20T17:57:26.208 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:26: run: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:26.208 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:27: run: shift 2026-02-20T17:57:26.208 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:29: run: export CEPH_MON=127.0.0.1:17110 2026-02-20T17:57:26.208 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:29: run: CEPH_MON=127.0.0.1:17110 2026-02-20T17:57:26.208 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:30: run: export CEPH_ARGS 2026-02-20T17:57:26.208 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:31: run: uuidgen 2026-02-20T17:57:26.208 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:31: run: CEPH_ARGS+='--fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none ' 2026-02-20T17:57:26.208 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:32: run: CEPH_ARGS+='--mon-host=127.0.0.1:17110 ' 2026-02-20T17:57:26.208 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:34: run: set 2026-02-20T17:57:26.208 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:34: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-20T17:57:26.210 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:34: run: local 'funcs=TEST_ec_profile_warning 2026-02-20T17:57:26.210 INFO:tasks.workunit.client.0.trial195.stderr:TEST_preload_no_warning 2026-02-20T17:57:26.210 INFO:tasks.workunit.client.0.trial195.stderr:TEST_preload_no_warning_default 2026-02-20T17:57:26.211 INFO:tasks.workunit.client.0.trial195.stderr:TEST_preload_warning' 2026-02-20T17:57:26.211 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-20T17:57:26.211 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_ec_profile_warning td/test-erasure-code-plugins 2026-02-20T17:57:26.211 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:92: TEST_ec_profile_warning: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:26.211 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:94: TEST_ec_profile_warning: setup td/test-erasure-code-plugins 2026-02-20T17:57:26.211 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:26.211 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-20T17:57:26.211 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:26.211 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T17:57:26.211 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-20T17:57:26.211 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T17:57:26.211 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T17:57:26.212 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T17:57:26.212 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T17:57:26.212 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T17:57:26.212 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T17:57:26.212 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T17:57:26.213 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T17:57:26.213 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T17:57:26.213 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T17:57:26.213 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T17:57:26.213 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T17:57:26.214 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:57:26.214 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T17:57:26.214 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T17:57:26.214 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:57:26.214 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T17:57:26.215 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T17:57:26.215 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-20T17:57:26.216 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T17:57:26.216 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:26.216 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:26.216 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.32971 2026-02-20T17:57:26.217 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T17:57:26.217 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T17:57:26.217 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-20T17:57:26.218 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-20T17:57:26.218 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:26.218 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:26.218 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.32971 2026-02-20T17:57:26.219 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-20T17:57:26.220 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-20T17:57:26.220 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-20T17:57:26.220 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-20T17:57:26.220 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-20T17:57:26.221 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:95: TEST_ec_profile_warning: run_mon td/test-erasure-code-plugins a 2026-02-20T17:57:26.221 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:26.221 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-20T17:57:26.221 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-20T17:57:26.221 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-20T17:57:26.221 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-20T17:57:26.221 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins 2026-02-20T17:57:26.249 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-20T17:57:26.249 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:57:26.249 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:57:26.249 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:57:26.249 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:26.250 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:26.250 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:26.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-20T17:57:26.275 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-20T17:57:26.277 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-20T17:57:26.277 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T17:57:26.277 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T17:57:26.277 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-20T17:57:26.277 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-20T17:57:26.277 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T17:57:26.277 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:57:26.277 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:57:26.278 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:26.278 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:26.278 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:26.278 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:57:26.278 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T17:57:26.279 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.32971/ceph-mon.a.asok config get fsid 2026-02-20T17:57:26.323 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-20T17:57:26.323 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T17:57:26.323 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T17:57:26.323 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-20T17:57:26.323 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-20T17:57:26.323 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T17:57:26.323 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:57:26.323 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:57:26.323 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:26.323 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:26.324 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:26.324 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:57:26.324 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T17:57:26.324 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.32971/ceph-mon.a.asok config get mon_host 2026-02-20T17:57:26.368 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:96: TEST_ec_profile_warning: run_mgr td/test-erasure-code-plugins x 2026-02-20T17:57:26.368 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:26.368 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-20T17:57:26.368 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-20T17:57:26.368 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-20T17:57:26.369 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-20T17:57:26.369 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-20T17:57:26.469 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-20T17:57:26.469 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:57:26.469 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:57:26.469 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:57:26.470 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:26.470 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:26.470 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:26.470 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T17:57:26.471 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T17:57:26.487 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:97: TEST_ec_profile_warning: seq 0 2 2026-02-20T17:57:26.490 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:97: TEST_ec_profile_warning: for id in $(seq 0 2) 2026-02-20T17:57:26.490 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:98: TEST_ec_profile_warning: run_osd td/test-erasure-code-plugins 0 2026-02-20T17:57:26.490 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:26.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T17:57:26.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-20T17:57:26.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T17:57:26.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-20T17:57:26.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-20T17:57:26.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T17:57:26.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T17:57:26.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T17:57:26.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-20T17:57:26.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-20T17:57:26.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T17:57:26.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T17:57:26.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-20T17:57:26.492 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T17:57:26.492 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:57:26.492 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:57:26.492 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:57:26.492 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:26.492 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:26.492 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:26.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:26.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T17:57:26.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T17:57:26.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T17:57:26.493 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-20T17:57:26.493 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-20T17:57:26.493 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T17:57:26.493 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T17:57:26.493 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T17:57:26.493 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T17:57:26.493 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T17:57:26.493 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T17:57:26.493 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-20T17:57:26.493 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T17:57:26.494 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5b2a3b2b-d3aa-4916-9141-ab4c87a1d562 2026-02-20T17:57:26.494 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 5b2a3b2b-d3aa-4916-9141-ab4c87a1d562' 2026-02-20T17:57:26.494 INFO:tasks.workunit.client.0.trial195.stdout:add osd0 5b2a3b2b-d3aa-4916-9141-ab4c87a1d562 2026-02-20T17:57:26.495 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T17:57:26.502 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCGoJhpkWDOHRAA5kmkXh2il+2qZ3XVJ7nyQA== 2026-02-20T17:57:26.502 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCGoJhpkWDOHRAA5kmkXh2il+2qZ3XVJ7nyQA=="}' 2026-02-20T17:57:26.502 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5b2a3b2b-d3aa-4916-9141-ab4c87a1d562 -i td/test-erasure-code-plugins/0/new.json 2026-02-20T17:57:26.606 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T17:57:26.612 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-20T17:57:26.613 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCGoJhpkWDOHRAA5kmkXh2il+2qZ3XVJ7nyQA== --osd-uuid 5b2a3b2b-d3aa-4916-9141-ab4c87a1d562 2026-02-20T17:57:26.630 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:26.627+0000 7f233daa1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:26.632 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:26.629+0000 7f233daa1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:26.632 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:26.630+0000 7f233daa1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:26.633 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:26.630+0000 7f233daa1900 -1 bdev(0x5633d9298c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-20T17:57:26.633 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:26.630+0000 7f233daa1900 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-20T17:57:27.041 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-20T17:57:27.041 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T17:57:27.042 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-20T17:57:27.042 INFO:tasks.workunit.client.0.trial195.stdout:adding osd0 key to auth repository 2026-02-20T17:57:27.042 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T17:57:27.146 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-20T17:57:27.147 INFO:tasks.workunit.client.0.trial195.stdout:start osd.0 2026-02-20T17:57:27.148 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T17:57:27.148 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T17:57:27.148 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T17:57:27.148 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T17:57:27.162 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:27.160+0000 7fae25593900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:27.164 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:27.161+0000 7fae25593900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:27.165 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:27.162+0000 7fae25593900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:27.245 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-20T17:57:27.246 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T17:57:27.246 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-20T17:57:27.246 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T17:57:27.246 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T17:57:27.247 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:57:27.247 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T17:57:27.247 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T17:57:27.248 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:57:27.248 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T17:57:27.341 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T17:57:27.385 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:27.383+0000 7fae25593900 -1 Falling back to public interface 2026-02-20T17:57:27.739 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:27.736+0000 7fae25593900 -1 osd.0 0 log_to_monitors true 2026-02-20T17:57:28.342 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T17:57:28.343 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:57:28.343 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T17:57:28.344 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T17:57:28.344 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:57:28.344 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T17:57:28.440 INFO:tasks.workunit.client.0.trial195.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3815955345,v1:127.0.0.1:6801/3815955345] [v2:127.0.0.1:6802/3815955345,v1:127.0.0.1:6803/3815955345] exists,up 5b2a3b2b-d3aa-4916-9141-ab4c87a1d562 2026-02-20T17:57:28.444 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T17:57:28.444 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T17:57:28.444 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T17:57:28.444 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:97: TEST_ec_profile_warning: for id in $(seq 0 2) 2026-02-20T17:57:28.444 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:98: TEST_ec_profile_warning: run_osd td/test-erasure-code-plugins 1 2026-02-20T17:57:28.444 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:28.444 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T17:57:28.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-20T17:57:28.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T17:57:28.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/1 2026-02-20T17:57:28.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-20T17:57:28.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T17:57:28.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T17:57:28.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T17:57:28.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/1' 2026-02-20T17:57:28.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/1/journal' 2026-02-20T17:57:28.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T17:57:28.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T17:57:28.446 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-20T17:57:28.446 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T17:57:28.446 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:57:28.446 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:57:28.446 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:57:28.446 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:28.446 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:28.446 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:28.446 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:28.446 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T17:57:28.446 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T17:57:28.446 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T17:57:28.447 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-20T17:57:28.447 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-20T17:57:28.447 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T17:57:28.447 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T17:57:28.447 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T17:57:28.447 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T17:57:28.447 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T17:57:28.447 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T17:57:28.447 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/1 2026-02-20T17:57:28.447 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T17:57:28.447 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4f748b8e-d3df-4412-a4a7-ed00619ad63c 2026-02-20T17:57:28.447 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 4f748b8e-d3df-4412-a4a7-ed00619ad63c' 2026-02-20T17:57:28.448 INFO:tasks.workunit.client.0.trial195.stdout:add osd1 4f748b8e-d3df-4412-a4a7-ed00619ad63c 2026-02-20T17:57:28.448 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T17:57:28.454 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCIoJhpPa/6GhAA6vhQYF/BWMFXlGjzkzcjyg== 2026-02-20T17:57:28.455 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCIoJhpPa/6GhAA6vhQYF/BWMFXlGjzkzcjyg=="}' 2026-02-20T17:57:28.455 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4f748b8e-d3df-4412-a4a7-ed00619ad63c -i td/test-erasure-code-plugins/1/new.json 2026-02-20T17:57:28.557 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T17:57:28.564 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/1/new.json 2026-02-20T17:57:28.565 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/1 --osd-journal=td/test-erasure-code-plugins/1/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCIoJhpPa/6GhAA6vhQYF/BWMFXlGjzkzcjyg== --osd-uuid 4f748b8e-d3df-4412-a4a7-ed00619ad63c 2026-02-20T17:57:28.581 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:28.579+0000 7fecea181900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:28.582 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:28.580+0000 7fecea181900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:28.583 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:28.581+0000 7fecea181900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:28.584 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:28.581+0000 7fecea181900 -1 bdev(0x557587394c00 td/test-erasure-code-plugins/1/block) open stat got: (1) Operation not permitted 2026-02-20T17:57:28.584 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:28.581+0000 7fecea181900 -1 bluestore(td/test-erasure-code-plugins/1) _read_fsid unparsable uuid 2026-02-20T17:57:28.982 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/1/keyring 2026-02-20T17:57:28.982 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T17:57:28.983 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-20T17:57:28.983 INFO:tasks.workunit.client.0.trial195.stdout:adding osd1 key to auth repository 2026-02-20T17:57:28.984 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T17:57:29.299 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-20T17:57:29.299 INFO:tasks.workunit.client.0.trial195.stdout:start osd.1 2026-02-20T17:57:29.299 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/1 --osd-journal=td/test-erasure-code-plugins/1/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T17:57:29.299 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T17:57:29.299 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T17:57:29.299 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T17:57:29.315 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:29.313+0000 7f9da2d86900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:29.316 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:29.314+0000 7f9da2d86900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:29.317 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:29.315+0000 7f9da2d86900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:29.500 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:29.498+0000 7f9da2d86900 -1 Falling back to public interface 2026-02-20T17:57:29.521 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-20T17:57:29.521 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T17:57:29.521 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-20T17:57:29.521 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T17:57:29.521 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T17:57:29.521 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:57:29.521 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T17:57:29.522 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T17:57:29.522 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:57:29.522 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T17:57:29.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T17:57:29.860 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:29.857+0000 7f9da2d86900 -1 osd.1 0 log_to_monitors true 2026-02-20T17:57:30.745 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T17:57:30.745 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:57:30.745 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T17:57:30.745 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T17:57:30.745 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:57:30.745 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T17:57:30.970 INFO:tasks.workunit.client.0.trial195.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1352907301,v1:127.0.0.1:6811/1352907301] [v2:127.0.0.1:6812/1352907301,v1:127.0.0.1:6813/1352907301] exists,up 4f748b8e-d3df-4412-a4a7-ed00619ad63c 2026-02-20T17:57:30.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T17:57:30.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T17:57:30.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T17:57:30.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:97: TEST_ec_profile_warning: for id in $(seq 0 2) 2026-02-20T17:57:30.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:98: TEST_ec_profile_warning: run_osd td/test-erasure-code-plugins 2 2026-02-20T17:57:30.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:30.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T17:57:30.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-20T17:57:30.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T17:57:30.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/2 2026-02-20T17:57:30.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-20T17:57:30.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T17:57:30.976 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T17:57:30.976 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T17:57:30.976 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/2' 2026-02-20T17:57:30.976 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/2/journal' 2026-02-20T17:57:30.976 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T17:57:30.976 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T17:57:30.976 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-20T17:57:30.976 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T17:57:30.976 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:57:30.976 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:57:30.976 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:57:30.976 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:30.977 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:30.977 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:30.977 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:30.977 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T17:57:30.977 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T17:57:30.977 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T17:57:30.977 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-20T17:57:30.977 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-20T17:57:30.977 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T17:57:30.977 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T17:57:30.977 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T17:57:30.977 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T17:57:30.978 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T17:57:30.978 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T17:57:30.978 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/2 2026-02-20T17:57:30.978 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T17:57:30.978 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=3ba3922d-8753-4145-946e-e70f63330aa1 2026-02-20T17:57:30.978 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 3ba3922d-8753-4145-946e-e70f63330aa1' 2026-02-20T17:57:30.978 INFO:tasks.workunit.client.0.trial195.stdout:add osd2 3ba3922d-8753-4145-946e-e70f63330aa1 2026-02-20T17:57:30.979 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T17:57:30.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCKoJhpq9aYOhAAFKONbyh2Yp9WQoDQGU778g== 2026-02-20T17:57:30.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCKoJhpq9aYOhAAFKONbyh2Yp9WQoDQGU778g=="}' 2026-02-20T17:57:30.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 3ba3922d-8753-4145-946e-e70f63330aa1 -i td/test-erasure-code-plugins/2/new.json 2026-02-20T17:57:31.213 INFO:tasks.workunit.client.0.trial195.stdout:2 2026-02-20T17:57:31.221 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/2/new.json 2026-02-20T17:57:31.222 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/2 --osd-journal=td/test-erasure-code-plugins/2/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCKoJhpq9aYOhAAFKONbyh2Yp9WQoDQGU778g== --osd-uuid 3ba3922d-8753-4145-946e-e70f63330aa1 2026-02-20T17:57:31.238 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:31.236+0000 7f90eb3e0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:31.240 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:31.237+0000 7f90eb3e0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:31.241 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:31.238+0000 7f90eb3e0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:31.241 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:31.239+0000 7f90eb3e0900 -1 bdev(0x55fe31524c00 td/test-erasure-code-plugins/2/block) open stat got: (1) Operation not permitted 2026-02-20T17:57:31.241 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:31.239+0000 7f90eb3e0900 -1 bluestore(td/test-erasure-code-plugins/2) _read_fsid unparsable uuid 2026-02-20T17:57:31.553 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/2/keyring 2026-02-20T17:57:31.553 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T17:57:31.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-20T17:57:31.554 INFO:tasks.workunit.client.0.trial195.stdout:adding osd2 key to auth repository 2026-02-20T17:57:31.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T17:57:31.872 INFO:tasks.workunit.client.0.trial195.stdout:start osd.2 2026-02-20T17:57:31.873 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-20T17:57:31.873 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/2 --osd-journal=td/test-erasure-code-plugins/2/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T17:57:31.873 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T17:57:31.873 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T17:57:31.874 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T17:57:31.888 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:31.886+0000 7fa8dfc91900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:31.889 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:31.887+0000 7fa8dfc91900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:31.890 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:31.888+0000 7fa8dfc91900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:32.064 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:32.061+0000 7fa8dfc91900 -1 Falling back to public interface 2026-02-20T17:57:32.098 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-20T17:57:32.098 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T17:57:32.098 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-20T17:57:32.099 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T17:57:32.099 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T17:57:32.099 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:57:32.099 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T17:57:32.099 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T17:57:32.100 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:57:32.100 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T17:57:32.320 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T17:57:32.359 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:32.357+0000 7fa8dfc91900 -1 osd.2 0 log_to_monitors true 2026-02-20T17:57:33.321 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T17:57:33.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:57:33.322 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T17:57:33.323 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T17:57:33.323 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:57:33.323 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T17:57:33.547 INFO:tasks.workunit.client.0.trial195.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1918716157,v1:127.0.0.1:6819/1918716157] [v2:127.0.0.1:6820/1918716157,v1:127.0.0.1:6821/1918716157] exists,up 3ba3922d-8753-4145-946e-e70f63330aa1 2026-02-20T17:57:33.548 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T17:57:33.549 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T17:57:33.549 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T17:57:33.549 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:100: TEST_ec_profile_warning: create_rbd_pool 2026-02-20T17:57:33.549 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-20T17:57:33.764 INFO:tasks.workunit.client.0.trial195.stderr:pool 'rbd' does not exist 2026-02-20T17:57:33.771 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-20T17:57:33.771 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-20T17:57:34.017 INFO:tasks.workunit.client.0.trial195.stderr:pool 'rbd' already exists 2026-02-20T17:57:34.025 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-20T17:57:35.026 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-20T17:57:35.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:101: TEST_ec_profile_warning: wait_for_clean 2026-02-20T17:57:35.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T17:57:35.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T17:57:35.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T17:57:35.310 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T17:57:35.310 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T17:57:35.310 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T17:57:35.311 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T17:57:35.311 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T17:57:35.311 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T17:57:35.392 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T17:57:35.392 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T17:57:35.392 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T17:57:35.392 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T17:57:35.392 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T17:57:35.392 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T17:57:35.612 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T17:57:35.613 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T17:57:35.613 INFO:tasks.workunit.client.0.trial195.stderr:2' 2026-02-20T17:57:35.613 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T17:57:35.613 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:57:35.613 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T17:57:35.675 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836484 2026-02-20T17:57:35.675 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836484 2026-02-20T17:57:35.675 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484' 2026-02-20T17:57:35.675 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:57:35.675 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T17:57:35.737 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705667 2026-02-20T17:57:35.737 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705667 2026-02-20T17:57:35.737 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484 1-38654705667' 2026-02-20T17:57:35.737 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:57:35.737 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T17:57:35.801 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542148 2026-02-20T17:57:35.801 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542148 2026-02-20T17:57:35.801 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484 1-38654705667 2-60129542148' 2026-02-20T17:57:35.801 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:57:35.802 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836484 2026-02-20T17:57:35.802 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:57:35.802 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T17:57:35.803 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836484 2026-02-20T17:57:35.803 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:57:35.804 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836484 2026-02-20T17:57:35.804 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-20T17:57:35.805 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.0 seq 21474836484 2026-02-20T17:57:35.805 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T17:57:36.028 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-20T17:57:36.028 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T17:57:37.029 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T17:57:37.029 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T17:57:37.253 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-20T17:57:37.254 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:57:37.254 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705667 2026-02-20T17:57:37.254 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:57:37.255 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T17:57:37.256 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705667 2026-02-20T17:57:37.256 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:57:37.257 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705667 2026-02-20T17:57:37.257 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705667' 2026-02-20T17:57:37.257 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.1 seq 38654705667 2026-02-20T17:57:37.258 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T17:57:37.481 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705668 -lt 38654705667 2026-02-20T17:57:37.481 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:57:37.481 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542148 2026-02-20T17:57:37.481 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:57:37.482 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T17:57:37.483 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542148 2026-02-20T17:57:37.483 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:57:37.484 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542148 2026-02-20T17:57:37.484 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542148' 2026-02-20T17:57:37.484 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.2 seq 60129542148 2026-02-20T17:57:37.484 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T17:57:37.706 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542148 -lt 60129542148 2026-02-20T17:57:37.707 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T17:57:37.707 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T17:57:37.707 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T17:57:38.017 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-20T17:57:38.017 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T17:57:38.017 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T17:57:38.017 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T17:57:38.017 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T17:57:38.017 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T17:57:38.017 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T17:57:38.017 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T17:57:38.235 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-20T17:57:38.236 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T17:57:38.236 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T17:57:38.236 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T17:57:38.546 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-20T17:57:38.547 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T17:57:38.547 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T17:57:38.547 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:103: TEST_ec_profile_warning: for plugin in ${legacy_jerasure_plugins[*]} 2026-02-20T17:57:38.547 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:104: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-jerasure_generic crush-failure-domain=osd technique=reed_sol_van plugin=jerasure_generic 2026-02-20T17:57:38.835 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-20T17:57:38.836 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:57:38.836 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:57:38.836 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:38.836 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:38.836 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:38.836 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:57:38.836 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-20T17:57:38.836 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.32971/ceph-mon.a.asok log flush 2026-02-20T17:57:38.875 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T17:57:38.878 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:106: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-jerasure_generic uses plugin jerasure_generic' td/test-erasure-code-plugins/mon.a.log 2026-02-20T17:57:38.880 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T17:57:38.708+0000 7f4bc0b5c640 0 mon.a@0(leader).osd e21 WARNING: erasure coding profile prof-jerasure_generic uses plugin jerasure_generic that has been deprecated. Please use jerasure instead. 2026-02-20T17:57:38.880 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T17:57:38.824+0000 7f4bc0b5c640 0 mon.a@0(leader).osd e22 WARNING: erasure coding profile prof-jerasure_generic uses plugin jerasure_generic that has been deprecated. Please use jerasure instead. 2026-02-20T17:57:38.880 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T17:57:38.824+0000 7f4bc0b5c640 0 mon.a@0(leader).osd e22 WARNING: erasure coding profile prof-jerasure_generic uses plugin jerasure_generic that has been deprecated. Please use jerasure instead. 2026-02-20T17:57:38.880 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:103: TEST_ec_profile_warning: for plugin in ${legacy_jerasure_plugins[*]} 2026-02-20T17:57:38.881 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:104: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-jerasure_sse3 crush-failure-domain=osd technique=reed_sol_van plugin=jerasure_sse3 2026-02-20T17:57:39.165 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-20T17:57:39.165 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:57:39.165 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:57:39.165 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:39.166 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:39.166 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:39.166 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:57:39.166 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-20T17:57:39.166 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.32971/ceph-mon.a.asok log flush 2026-02-20T17:57:39.203 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T17:57:39.206 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:106: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-jerasure_sse3 uses plugin jerasure_sse3' td/test-erasure-code-plugins/mon.a.log 2026-02-20T17:57:39.208 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T17:57:39.037+0000 7f4bc0b5c640 0 mon.a@0(leader).osd e22 WARNING: erasure coding profile prof-jerasure_sse3 uses plugin jerasure_sse3 that has been deprecated. Please use jerasure instead. 2026-02-20T17:57:39.208 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T17:57:39.153+0000 7f4bc0b5c640 0 mon.a@0(leader).osd e23 WARNING: erasure coding profile prof-jerasure_sse3 uses plugin jerasure_sse3 that has been deprecated. Please use jerasure instead. 2026-02-20T17:57:39.208 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T17:57:39.153+0000 7f4bc0b5c640 0 mon.a@0(leader).osd e23 WARNING: erasure coding profile prof-jerasure_sse3 uses plugin jerasure_sse3 that has been deprecated. Please use jerasure instead. 2026-02-20T17:57:39.209 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:103: TEST_ec_profile_warning: for plugin in ${legacy_jerasure_plugins[*]} 2026-02-20T17:57:39.209 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:104: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-jerasure_sse4 crush-failure-domain=osd technique=reed_sol_van plugin=jerasure_sse4 2026-02-20T17:57:39.497 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-20T17:57:39.497 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:57:39.498 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:57:39.498 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:39.498 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:39.498 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:39.498 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:57:39.498 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-20T17:57:39.498 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.32971/ceph-mon.a.asok log flush 2026-02-20T17:57:39.538 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T17:57:39.542 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:106: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-jerasure_sse4 uses plugin jerasure_sse4' td/test-erasure-code-plugins/mon.a.log 2026-02-20T17:57:39.545 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T17:57:39.369+0000 7f4bc0b5c640 0 mon.a@0(leader).osd e23 WARNING: erasure coding profile prof-jerasure_sse4 uses plugin jerasure_sse4 that has been deprecated. Please use jerasure instead. 2026-02-20T17:57:39.545 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T17:57:39.486+0000 7f4bc0b5c640 0 mon.a@0(leader).osd e24 WARNING: erasure coding profile prof-jerasure_sse4 uses plugin jerasure_sse4 that has been deprecated. Please use jerasure instead. 2026-02-20T17:57:39.545 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T17:57:39.486+0000 7f4bc0b5c640 0 mon.a@0(leader).osd e24 WARNING: erasure coding profile prof-jerasure_sse4 uses plugin jerasure_sse4 that has been deprecated. Please use jerasure instead. 2026-02-20T17:57:39.545 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:109: TEST_ec_profile_warning: for plugin in ${legacy_shec_plugins[*]} 2026-02-20T17:57:39.545 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:110: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-shec_generic crush-failure-domain=osd plugin=shec_generic 2026-02-20T17:57:39.833 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-20T17:57:39.833 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:57:39.833 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:57:39.834 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:39.834 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:39.834 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:39.834 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:57:39.834 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-20T17:57:39.834 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.32971/ceph-mon.a.asok log flush 2026-02-20T17:57:39.875 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T17:57:39.879 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:112: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-shec_generic uses plugin shec_generic' td/test-erasure-code-plugins/mon.a.log 2026-02-20T17:57:39.881 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T17:57:39.706+0000 7f4bc0b5c640 0 mon.a@0(leader).osd e24 WARNING: erasure coding profile prof-shec_generic uses plugin shec_generic that has been deprecated. Please use shec instead. 2026-02-20T17:57:39.881 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T17:57:39.823+0000 7f4bc0b5c640 0 mon.a@0(leader).osd e25 WARNING: erasure coding profile prof-shec_generic uses plugin shec_generic that has been deprecated. Please use shec instead. 2026-02-20T17:57:39.881 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T17:57:39.823+0000 7f4bc0b5c640 0 mon.a@0(leader).osd e25 WARNING: erasure coding profile prof-shec_generic uses plugin shec_generic that has been deprecated. Please use shec instead. 2026-02-20T17:57:39.881 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:109: TEST_ec_profile_warning: for plugin in ${legacy_shec_plugins[*]} 2026-02-20T17:57:39.882 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:110: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-shec_sse3 crush-failure-domain=osd plugin=shec_sse3 2026-02-20T17:57:40.171 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-20T17:57:40.172 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:57:40.172 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:57:40.172 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:40.172 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:40.172 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:40.172 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:57:40.172 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-20T17:57:40.172 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.32971/ceph-mon.a.asok log flush 2026-02-20T17:57:40.212 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T17:57:40.217 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:112: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-shec_sse3 uses plugin shec_sse3' td/test-erasure-code-plugins/mon.a.log 2026-02-20T17:57:40.219 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T17:57:40.044+0000 7f4bc0b5c640 0 mon.a@0(leader).osd e25 WARNING: erasure coding profile prof-shec_sse3 uses plugin shec_sse3 that has been deprecated. Please use shec instead. 2026-02-20T17:57:40.220 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T17:57:40.160+0000 7f4bc0b5c640 0 mon.a@0(leader).osd e26 WARNING: erasure coding profile prof-shec_sse3 uses plugin shec_sse3 that has been deprecated. Please use shec instead. 2026-02-20T17:57:40.220 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T17:57:40.160+0000 7f4bc0b5c640 0 mon.a@0(leader).osd e26 WARNING: erasure coding profile prof-shec_sse3 uses plugin shec_sse3 that has been deprecated. Please use shec instead. 2026-02-20T17:57:40.220 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:109: TEST_ec_profile_warning: for plugin in ${legacy_shec_plugins[*]} 2026-02-20T17:57:40.220 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:110: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-shec_sse4 crush-failure-domain=osd plugin=shec_sse4 2026-02-20T17:57:40.503 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-20T17:57:40.503 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:57:40.503 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:57:40.503 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:40.503 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:40.503 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:40.503 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:57:40.503 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-20T17:57:40.503 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.32971/ceph-mon.a.asok log flush 2026-02-20T17:57:40.544 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T17:57:40.548 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:112: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-shec_sse4 uses plugin shec_sse4' td/test-erasure-code-plugins/mon.a.log 2026-02-20T17:57:40.551 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T17:57:40.377+0000 7f4bc0b5c640 0 mon.a@0(leader).osd e26 WARNING: erasure coding profile prof-shec_sse4 uses plugin shec_sse4 that has been deprecated. Please use shec instead. 2026-02-20T17:57:40.551 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T17:57:40.493+0000 7f4bc0b5c640 0 mon.a@0(leader).osd e27 WARNING: erasure coding profile prof-shec_sse4 uses plugin shec_sse4 that has been deprecated. Please use shec instead. 2026-02-20T17:57:40.551 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T17:57:40.493+0000 7f4bc0b5c640 0 mon.a@0(leader).osd e27 WARNING: erasure coding profile prof-shec_sse4 uses plugin shec_sse4 that has been deprecated. Please use shec instead. 2026-02-20T17:57:40.552 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:115: TEST_ec_profile_warning: teardown td/test-erasure-code-plugins 2026-02-20T17:57:40.552 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:40.552 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T17:57:40.552 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-20T17:57:40.553 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T17:57:40.553 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T17:57:40.553 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T17:57:40.553 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T17:57:40.553 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T17:57:40.659 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T17:57:40.659 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T17:57:40.659 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T17:57:40.660 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T17:57:40.660 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T17:57:40.661 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T17:57:40.661 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T17:57:40.661 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:57:40.661 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T17:57:40.662 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T17:57:40.662 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:57:40.662 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T17:57:40.663 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T17:57:40.663 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-20T17:57:40.669 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T17:57:40.670 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:40.670 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:40.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.32971 2026-02-20T17:57:40.672 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T17:57:40.672 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T17:57:40.672 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-20T17:57:40.672 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_preload_no_warning td/test-erasure-code-plugins 2026-02-20T17:57:40.672 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:58: TEST_preload_no_warning: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:40.672 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:60: TEST_preload_no_warning: for plugin in ${plugins[*]} 2026-02-20T17:57:40.672 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:61: TEST_preload_no_warning: setup td/test-erasure-code-plugins 2026-02-20T17:57:40.673 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:40.673 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-20T17:57:40.673 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:40.673 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T17:57:40.673 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-20T17:57:40.673 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T17:57:40.673 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T17:57:40.673 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T17:57:40.673 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T17:57:40.673 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T17:57:40.674 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T17:57:40.674 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T17:57:40.674 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T17:57:40.674 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T17:57:40.675 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T17:57:40.675 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T17:57:40.675 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T17:57:40.676 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:57:40.676 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T17:57:40.676 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T17:57:40.677 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:57:40.677 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T17:57:40.678 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T17:57:40.678 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-20T17:57:40.679 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T17:57:40.679 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:40.679 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:40.679 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.32971 2026-02-20T17:57:40.680 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T17:57:40.680 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T17:57:40.680 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-20T17:57:40.681 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-20T17:57:40.681 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:40.681 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:40.681 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.32971 2026-02-20T17:57:40.682 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-20T17:57:40.683 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-20T17:57:40.683 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-20T17:57:40.684 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-20T17:57:40.684 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:62: TEST_preload_no_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=jerasure 2026-02-20T17:57:40.684 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:40.684 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-20T17:57:40.684 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-20T17:57:40.684 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-20T17:57:40.684 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-20T17:57:40.684 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=jerasure 2026-02-20T17:57:40.703 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-20T17:57:40.703 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:57:40.703 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:57:40.703 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:57:40.703 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:40.703 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:40.703 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:40.703 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=jerasure 2026-02-20T17:57:40.728 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-20T17:57:40.730 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-20T17:57:40.730 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T17:57:40.730 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T17:57:40.730 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-20T17:57:40.730 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-20T17:57:40.730 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T17:57:40.730 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:57:40.730 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:57:40.731 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:40.731 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:40.731 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:40.731 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:57:40.731 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T17:57:40.732 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.32971/ceph-mon.a.asok config get fsid 2026-02-20T17:57:40.776 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-20T17:57:40.776 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T17:57:40.776 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T17:57:40.776 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-20T17:57:40.776 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-20T17:57:40.776 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T17:57:40.776 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:57:40.776 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:57:40.776 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:40.777 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:40.777 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:40.777 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:57:40.777 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T17:57:40.777 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.32971/ceph-mon.a.asok config get mon_host 2026-02-20T17:57:40.819 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:63: TEST_preload_no_warning: run_mgr td/test-erasure-code-plugins x 2026-02-20T17:57:40.819 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:40.819 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-20T17:57:40.819 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-20T17:57:40.820 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-20T17:57:40.820 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-20T17:57:40.820 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-20T17:57:40.921 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-20T17:57:40.921 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:57:40.922 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:57:40.922 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:57:40.922 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:40.922 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:40.922 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:40.922 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T17:57:40.922 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T17:57:40.938 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: get_asok_path mon.a 2026-02-20T17:57:40.938 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:57:40.938 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:57:40.938 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:40.938 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:40.938 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:40.939 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:57:40.939 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: CEPH_ARGS= 2026-02-20T17:57:40.939 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.32971/ceph-mon.a.asok log flush 2026-02-20T17:57:40.977 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T17:57:40.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:65: TEST_preload_no_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=jerasure 2026-02-20T17:57:40.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:40.986 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T17:57:40.986 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-20T17:57:40.986 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T17:57:40.986 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-20T17:57:40.986 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-20T17:57:40.986 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T17:57:40.986 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T17:57:40.986 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T17:57:40.986 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-20T17:57:40.986 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-20T17:57:40.987 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T17:57:40.987 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T17:57:40.987 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-20T17:57:40.987 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T17:57:40.987 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:57:40.987 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:57:40.987 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:57:40.987 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:40.987 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:40.987 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:40.987 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:40.987 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T17:57:40.988 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T17:57:40.988 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T17:57:40.988 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-20T17:57:40.988 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-20T17:57:40.988 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T17:57:40.988 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T17:57:40.988 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T17:57:40.988 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T17:57:40.988 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T17:57:40.989 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=jerasure 2026-02-20T17:57:40.989 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-20T17:57:40.989 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T17:57:40.989 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5c69690b-503e-4e67-8884-4b8a6ade5871 2026-02-20T17:57:40.989 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 5c69690b-503e-4e67-8884-4b8a6ade5871' 2026-02-20T17:57:40.990 INFO:tasks.workunit.client.0.trial195.stdout:add osd0 5c69690b-503e-4e67-8884-4b8a6ade5871 2026-02-20T17:57:40.990 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T17:57:40.997 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCUoJhpM0VOOxAAyhdGY9GGHm0mqS/gOopYhA== 2026-02-20T17:57:40.997 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCUoJhpM0VOOxAAyhdGY9GGHm0mqS/gOopYhA=="}' 2026-02-20T17:57:40.997 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5c69690b-503e-4e67-8884-4b8a6ade5871 -i td/test-erasure-code-plugins/0/new.json 2026-02-20T17:57:41.100 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T17:57:41.106 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-20T17:57:41.107 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure --mkfs --key AQCUoJhpM0VOOxAAyhdGY9GGHm0mqS/gOopYhA== --osd-uuid 5c69690b-503e-4e67-8884-4b8a6ade5871 2026-02-20T17:57:41.125 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:41.123+0000 7f4df75e0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:41.126 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:41.125+0000 7f4df75e0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:41.127 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:41.126+0000 7f4df75e0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:41.128 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:41.126+0000 7f4df75e0900 -1 bdev(0x55b542506c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-20T17:57:41.128 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:41.126+0000 7f4df75e0900 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-20T17:57:41.563 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-20T17:57:41.563 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T17:57:41.564 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-20T17:57:41.564 INFO:tasks.workunit.client.0.trial195.stdout:adding osd0 key to auth repository 2026-02-20T17:57:41.565 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T17:57:41.666 INFO:tasks.workunit.client.0.trial195.stdout:start osd.0 2026-02-20T17:57:41.667 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-20T17:57:41.667 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure 2026-02-20T17:57:41.667 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T17:57:41.667 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T17:57:41.667 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T17:57:41.683 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:41.682+0000 7f0041c7d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:41.684 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:41.683+0000 7f0041c7d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:41.685 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:41.684+0000 7f0041c7d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:41.763 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-20T17:57:41.763 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T17:57:41.763 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-20T17:57:41.763 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T17:57:41.763 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T17:57:41.763 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:57:41.763 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T17:57:41.763 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T17:57:41.764 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:57:41.764 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T17:57:41.858 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T17:57:41.910 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:41.908+0000 7f0041c7d900 -1 Falling back to public interface 2026-02-20T17:57:42.228 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:42.226+0000 7f0041c7d900 -1 osd.0 0 log_to_monitors true 2026-02-20T17:57:42.860 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T17:57:42.860 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:57:42.860 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T17:57:42.861 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T17:57:42.861 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:57:42.861 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T17:57:42.957 INFO:tasks.workunit.client.0.trial195.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3468710590,v1:127.0.0.1:6801/3468710590] [v2:127.0.0.1:6802/3468710590,v1:127.0.0.1:6803/3468710590] exists,up 5c69690b-503e-4e67-8884-4b8a6ade5871 2026-02-20T17:57:42.959 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T17:57:42.959 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T17:57:42.959 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T17:57:42.959 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: get_asok_path osd.0 2026-02-20T17:57:42.959 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-20T17:57:42.959 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-20T17:57:42.959 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:42.960 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:42.960 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:42.960 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-osd.0.asok 2026-02-20T17:57:42.960 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: CEPH_ARGS= 2026-02-20T17:57:42.960 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.32971/ceph-osd.0.asok log flush 2026-02-20T17:57:43.004 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T17:57:43.008 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:67: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/mon.a.log 2026-02-20T17:57:43.009 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:68: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/osd.0.log 2026-02-20T17:57:43.011 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:69: TEST_preload_no_warning: teardown td/test-erasure-code-plugins 2026-02-20T17:57:43.012 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:43.012 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T17:57:43.012 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-20T17:57:43.012 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T17:57:43.012 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T17:57:43.012 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T17:57:43.012 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T17:57:43.012 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T17:57:43.118 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T17:57:43.118 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T17:57:43.119 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T17:57:43.119 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T17:57:43.121 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T17:57:43.121 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T17:57:43.121 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T17:57:43.122 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:57:43.122 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T17:57:43.123 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T17:57:43.123 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:57:43.123 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T17:57:43.124 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T17:57:43.125 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-20T17:57:43.128 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T17:57:43.128 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:43.128 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:43.128 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.32971 2026-02-20T17:57:43.131 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T17:57:43.132 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T17:57:43.132 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:60: TEST_preload_no_warning: for plugin in ${plugins[*]} 2026-02-20T17:57:43.132 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:61: TEST_preload_no_warning: setup td/test-erasure-code-plugins 2026-02-20T17:57:43.132 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:43.132 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-20T17:57:43.132 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:43.132 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T17:57:43.132 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-20T17:57:43.132 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T17:57:43.132 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T17:57:43.132 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T17:57:43.132 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T17:57:43.132 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T17:57:43.132 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T17:57:43.132 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T17:57:43.132 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T17:57:43.132 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T17:57:43.133 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T17:57:43.133 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T17:57:43.133 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T17:57:43.135 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:57:43.135 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T17:57:43.135 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T17:57:43.135 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:57:43.135 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T17:57:43.136 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T17:57:43.136 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-20T17:57:43.137 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T17:57:43.137 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:43.137 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:43.138 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.32971 2026-02-20T17:57:43.138 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T17:57:43.138 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T17:57:43.138 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-20T17:57:43.139 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-20T17:57:43.139 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:43.140 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:43.140 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.32971 2026-02-20T17:57:43.141 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-20T17:57:43.142 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-20T17:57:43.142 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-20T17:57:43.142 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-20T17:57:43.142 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:62: TEST_preload_no_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=shec 2026-02-20T17:57:43.142 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:43.142 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-20T17:57:43.142 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-20T17:57:43.142 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-20T17:57:43.142 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-20T17:57:43.143 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=shec 2026-02-20T17:57:43.161 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-20T17:57:43.161 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:57:43.161 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:57:43.161 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:57:43.161 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:43.161 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:43.162 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:43.162 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=shec 2026-02-20T17:57:43.187 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-20T17:57:43.188 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-20T17:57:43.189 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T17:57:43.189 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T17:57:43.189 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-20T17:57:43.189 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-20T17:57:43.190 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T17:57:43.190 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:57:43.190 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:57:43.190 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:43.191 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:43.191 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:43.191 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:57:43.191 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T17:57:43.191 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.32971/ceph-mon.a.asok config get fsid 2026-02-20T17:57:43.236 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-20T17:57:43.236 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T17:57:43.236 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T17:57:43.236 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-20T17:57:43.236 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-20T17:57:43.236 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T17:57:43.236 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:57:43.236 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:57:43.236 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:43.236 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:43.236 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:43.236 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:57:43.236 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T17:57:43.236 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.32971/ceph-mon.a.asok config get mon_host 2026-02-20T17:57:43.278 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:63: TEST_preload_no_warning: run_mgr td/test-erasure-code-plugins x 2026-02-20T17:57:43.278 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:43.278 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-20T17:57:43.278 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-20T17:57:43.279 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-20T17:57:43.279 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-20T17:57:43.279 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-20T17:57:43.381 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-20T17:57:43.381 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:57:43.382 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:57:43.382 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:57:43.382 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:43.382 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:43.382 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:43.382 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T17:57:43.383 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T17:57:43.400 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: get_asok_path mon.a 2026-02-20T17:57:43.400 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:57:43.400 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:57:43.400 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:43.401 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:43.401 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:43.401 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:57:43.401 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: CEPH_ARGS= 2026-02-20T17:57:43.401 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.32971/ceph-mon.a.asok log flush 2026-02-20T17:57:43.441 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T17:57:43.449 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:65: TEST_preload_no_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=shec 2026-02-20T17:57:43.449 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:43.449 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T17:57:43.449 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-20T17:57:43.449 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T17:57:43.449 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-20T17:57:43.449 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-20T17:57:43.449 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T17:57:43.449 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T17:57:43.450 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T17:57:43.450 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-20T17:57:43.450 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-20T17:57:43.450 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T17:57:43.450 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T17:57:43.450 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-20T17:57:43.450 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T17:57:43.450 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:57:43.450 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:57:43.450 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:57:43.450 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:43.450 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:43.451 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:43.451 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:43.451 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T17:57:43.451 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T17:57:43.451 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T17:57:43.451 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-20T17:57:43.451 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-20T17:57:43.451 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T17:57:43.451 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T17:57:43.451 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T17:57:43.451 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T17:57:43.451 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T17:57:43.451 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=shec 2026-02-20T17:57:43.452 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-20T17:57:43.452 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T17:57:43.452 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=88cc8c40-61f1-4d3d-8abc-a2bb5858c52a 2026-02-20T17:57:43.452 INFO:tasks.workunit.client.0.trial195.stdout:add osd0 88cc8c40-61f1-4d3d-8abc-a2bb5858c52a 2026-02-20T17:57:43.452 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 88cc8c40-61f1-4d3d-8abc-a2bb5858c52a' 2026-02-20T17:57:43.453 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T17:57:43.460 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCXoJhp0clQGxAAo+j0TCmYda2Y2fG0LQYaHQ== 2026-02-20T17:57:43.460 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCXoJhp0clQGxAAo+j0TCmYda2Y2fG0LQYaHQ=="}' 2026-02-20T17:57:43.460 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 88cc8c40-61f1-4d3d-8abc-a2bb5858c52a -i td/test-erasure-code-plugins/0/new.json 2026-02-20T17:57:43.563 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T17:57:43.570 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-20T17:57:43.571 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec --mkfs --key AQCXoJhp0clQGxAAo+j0TCmYda2Y2fG0LQYaHQ== --osd-uuid 88cc8c40-61f1-4d3d-8abc-a2bb5858c52a 2026-02-20T17:57:43.587 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:43.586+0000 7fc82bfa8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:43.589 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:43.588+0000 7fc82bfa8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:43.590 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:43.589+0000 7fc82bfa8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:43.591 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:43.589+0000 7fc82bfa8900 -1 bdev(0x555d7f376c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-20T17:57:43.591 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:43.589+0000 7fc82bfa8900 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-20T17:57:44.013 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-20T17:57:44.013 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T17:57:44.014 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-20T17:57:44.014 INFO:tasks.workunit.client.0.trial195.stdout:adding osd0 key to auth repository 2026-02-20T17:57:44.014 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T17:57:44.117 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-20T17:57:44.117 INFO:tasks.workunit.client.0.trial195.stdout:start osd.0 2026-02-20T17:57:44.118 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec 2026-02-20T17:57:44.118 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T17:57:44.118 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T17:57:44.118 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T17:57:44.133 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:44.132+0000 7f1de720c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:44.134 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:44.133+0000 7f1de720c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:44.136 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:44.134+0000 7f1de720c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:44.215 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-20T17:57:44.215 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T17:57:44.215 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-20T17:57:44.215 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T17:57:44.215 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T17:57:44.215 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:57:44.216 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T17:57:44.216 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T17:57:44.216 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:57:44.216 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T17:57:44.313 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T17:57:44.336 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:44.335+0000 7f1de720c900 -1 Falling back to public interface 2026-02-20T17:57:44.699 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:44.698+0000 7f1de720c900 -1 osd.0 0 log_to_monitors true 2026-02-20T17:57:45.315 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T17:57:45.315 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:57:45.315 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T17:57:45.316 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T17:57:45.316 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:57:45.316 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T17:57:45.414 INFO:tasks.workunit.client.0.trial195.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2931774062,v1:127.0.0.1:6801/2931774062] [v2:127.0.0.1:6802/2931774062,v1:127.0.0.1:6803/2931774062] exists,up 88cc8c40-61f1-4d3d-8abc-a2bb5858c52a 2026-02-20T17:57:45.415 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T17:57:45.415 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T17:57:45.415 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T17:57:45.415 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: get_asok_path osd.0 2026-02-20T17:57:45.415 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-20T17:57:45.415 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-20T17:57:45.415 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:45.415 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:45.415 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:45.415 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-osd.0.asok 2026-02-20T17:57:45.415 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: CEPH_ARGS= 2026-02-20T17:57:45.416 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.32971/ceph-osd.0.asok log flush 2026-02-20T17:57:45.463 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T17:57:45.468 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:67: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/mon.a.log 2026-02-20T17:57:45.469 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:68: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/osd.0.log 2026-02-20T17:57:45.470 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:69: TEST_preload_no_warning: teardown td/test-erasure-code-plugins 2026-02-20T17:57:45.470 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:45.470 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T17:57:45.470 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-20T17:57:45.471 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T17:57:45.471 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T17:57:45.471 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T17:57:45.471 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T17:57:45.471 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T17:57:45.578 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T17:57:45.578 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T17:57:45.579 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T17:57:45.579 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T17:57:45.580 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T17:57:45.580 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T17:57:45.580 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T17:57:45.580 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:57:45.581 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T17:57:45.581 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T17:57:45.581 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:57:45.582 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T17:57:45.582 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T17:57:45.583 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-20T17:57:45.586 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T17:57:45.586 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:45.586 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:45.586 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.32971 2026-02-20T17:57:45.587 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T17:57:45.587 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T17:57:45.587 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:60: TEST_preload_no_warning: for plugin in ${plugins[*]} 2026-02-20T17:57:45.587 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:61: TEST_preload_no_warning: setup td/test-erasure-code-plugins 2026-02-20T17:57:45.587 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:45.588 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-20T17:57:45.588 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:45.588 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T17:57:45.588 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-20T17:57:45.588 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T17:57:45.588 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T17:57:45.588 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T17:57:45.588 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T17:57:45.588 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T17:57:45.589 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T17:57:45.589 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T17:57:45.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T17:57:45.590 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T17:57:45.591 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T17:57:45.591 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T17:57:45.591 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T17:57:45.592 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:57:45.592 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T17:57:45.592 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T17:57:45.592 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:57:45.593 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T17:57:45.594 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T17:57:45.595 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-20T17:57:45.596 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T17:57:45.596 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:45.596 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:45.596 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.32971 2026-02-20T17:57:45.597 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T17:57:45.597 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T17:57:45.597 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-20T17:57:45.598 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-20T17:57:45.598 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:45.598 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:45.598 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.32971 2026-02-20T17:57:45.599 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-20T17:57:45.600 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-20T17:57:45.600 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-20T17:57:45.600 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-20T17:57:45.600 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:62: TEST_preload_no_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=lrc 2026-02-20T17:57:45.600 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:45.600 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-20T17:57:45.600 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-20T17:57:45.601 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-20T17:57:45.601 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-20T17:57:45.601 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=lrc 2026-02-20T17:57:45.620 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-20T17:57:45.620 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:57:45.620 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:57:45.620 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:57:45.620 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:45.620 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:45.620 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:45.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=lrc 2026-02-20T17:57:45.644 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-20T17:57:45.647 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-20T17:57:45.648 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T17:57:45.648 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T17:57:45.648 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-20T17:57:45.648 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-20T17:57:45.648 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T17:57:45.649 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:57:45.649 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:57:45.649 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:45.649 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:45.649 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:45.650 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:57:45.650 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T17:57:45.650 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.32971/ceph-mon.a.asok config get fsid 2026-02-20T17:57:45.688 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-20T17:57:45.689 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T17:57:45.689 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T17:57:45.689 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-20T17:57:45.689 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-20T17:57:45.689 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T17:57:45.689 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:57:45.689 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:57:45.690 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:45.690 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:45.690 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:45.690 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:57:45.690 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T17:57:45.690 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.32971/ceph-mon.a.asok config get mon_host 2026-02-20T17:57:45.734 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:63: TEST_preload_no_warning: run_mgr td/test-erasure-code-plugins x 2026-02-20T17:57:45.734 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:45.734 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-20T17:57:45.734 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-20T17:57:45.734 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-20T17:57:45.734 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-20T17:57:45.734 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-20T17:57:45.834 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-20T17:57:45.834 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:57:45.834 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:57:45.834 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:57:45.834 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:45.834 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:45.834 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:45.834 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T17:57:45.835 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T17:57:45.853 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: get_asok_path mon.a 2026-02-20T17:57:45.853 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:57:45.853 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:57:45.853 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:45.853 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:45.853 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:45.853 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:57:45.853 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: CEPH_ARGS= 2026-02-20T17:57:45.853 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.32971/ceph-mon.a.asok log flush 2026-02-20T17:57:45.893 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T17:57:45.903 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:65: TEST_preload_no_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=lrc 2026-02-20T17:57:45.903 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:45.903 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T17:57:45.903 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-20T17:57:45.903 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T17:57:45.903 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-20T17:57:45.903 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-20T17:57:45.903 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T17:57:45.904 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T17:57:45.904 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T17:57:45.904 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-20T17:57:45.904 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-20T17:57:45.904 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T17:57:45.904 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T17:57:45.904 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-20T17:57:45.904 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T17:57:45.904 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:57:45.904 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:57:45.904 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:57:45.904 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:45.905 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:45.905 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:45.905 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:45.905 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T17:57:45.905 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T17:57:45.905 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T17:57:45.905 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-20T17:57:45.905 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-20T17:57:45.905 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T17:57:45.905 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T17:57:45.905 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T17:57:45.905 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T17:57:45.905 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T17:57:45.906 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=lrc 2026-02-20T17:57:45.906 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-20T17:57:45.906 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T17:57:45.906 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=fa5cb9b7-ae9f-4348-9a71-e46d05de1a30 2026-02-20T17:57:45.906 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 fa5cb9b7-ae9f-4348-9a71-e46d05de1a30' 2026-02-20T17:57:45.906 INFO:tasks.workunit.client.0.trial195.stdout:add osd0 fa5cb9b7-ae9f-4348-9a71-e46d05de1a30 2026-02-20T17:57:45.907 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T17:57:45.912 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCZoJhpOBpFNhAAM96EyVrePPX6eUb3I+b48A== 2026-02-20T17:57:45.912 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCZoJhpOBpFNhAAM96EyVrePPX6eUb3I+b48A=="}' 2026-02-20T17:57:45.912 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new fa5cb9b7-ae9f-4348-9a71-e46d05de1a30 -i td/test-erasure-code-plugins/0/new.json 2026-02-20T17:57:46.015 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T17:57:46.022 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-20T17:57:46.023 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=lrc --mkfs --key AQCZoJhpOBpFNhAAM96EyVrePPX6eUb3I+b48A== --osd-uuid fa5cb9b7-ae9f-4348-9a71-e46d05de1a30 2026-02-20T17:57:46.040 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:46.039+0000 7f539b295900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:46.041 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:46.040+0000 7f539b295900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:46.042 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:46.041+0000 7f539b295900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:46.043 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:46.042+0000 7f539b295900 -1 bdev(0x55de08988c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-20T17:57:46.043 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:46.042+0000 7f539b295900 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-20T17:57:46.430 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-20T17:57:46.430 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T17:57:46.430 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-20T17:57:46.431 INFO:tasks.workunit.client.0.trial195.stdout:adding osd0 key to auth repository 2026-02-20T17:57:46.431 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T17:57:46.533 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-20T17:57:46.534 INFO:tasks.workunit.client.0.trial195.stdout:start osd.0 2026-02-20T17:57:46.535 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=lrc 2026-02-20T17:57:46.535 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T17:57:46.535 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T17:57:46.535 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T17:57:46.551 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:46.550+0000 7fa9f5272900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:46.552 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:46.551+0000 7fa9f5272900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:46.553 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:46.552+0000 7fa9f5272900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:46.630 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-20T17:57:46.630 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T17:57:46.630 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-20T17:57:46.630 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T17:57:46.630 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T17:57:46.631 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T17:57:46.631 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:57:46.631 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T17:57:46.631 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:57:46.632 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T17:57:46.726 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T17:57:46.745 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:46.744+0000 7fa9f5272900 -1 Falling back to public interface 2026-02-20T17:57:47.085 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:47.083+0000 7fa9f5272900 -1 osd.0 0 log_to_monitors true 2026-02-20T17:57:47.728 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T17:57:47.728 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:57:47.728 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T17:57:47.729 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T17:57:47.729 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:57:47.729 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T17:57:47.824 INFO:tasks.workunit.client.0.trial195.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/698252702,v1:127.0.0.1:6801/698252702] [v2:127.0.0.1:6802/698252702,v1:127.0.0.1:6803/698252702] exists,up fa5cb9b7-ae9f-4348-9a71-e46d05de1a30 2026-02-20T17:57:47.826 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T17:57:47.826 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T17:57:47.826 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T17:57:47.827 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: get_asok_path osd.0 2026-02-20T17:57:47.827 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-20T17:57:47.827 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-20T17:57:47.827 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:47.827 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:47.827 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:47.827 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-osd.0.asok 2026-02-20T17:57:47.827 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: CEPH_ARGS= 2026-02-20T17:57:47.827 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.32971/ceph-osd.0.asok log flush 2026-02-20T17:57:47.870 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T17:57:47.874 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:67: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/mon.a.log 2026-02-20T17:57:47.875 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:68: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/osd.0.log 2026-02-20T17:57:47.877 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:69: TEST_preload_no_warning: teardown td/test-erasure-code-plugins 2026-02-20T17:57:47.877 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:47.877 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T17:57:47.877 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-20T17:57:47.877 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T17:57:47.877 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T17:57:47.877 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T17:57:47.877 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T17:57:47.877 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T17:57:47.983 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T17:57:47.984 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T17:57:47.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T17:57:47.985 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T17:57:47.986 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T17:57:47.986 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T17:57:47.987 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T17:57:47.987 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:57:47.988 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T17:57:47.988 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T17:57:47.988 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:57:47.989 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T17:57:47.990 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T17:57:47.990 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-20T17:57:47.993 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T17:57:47.994 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:47.994 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:47.994 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.32971 2026-02-20T17:57:47.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T17:57:47.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T17:57:47.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:60: TEST_preload_no_warning: for plugin in ${plugins[*]} 2026-02-20T17:57:47.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:61: TEST_preload_no_warning: setup td/test-erasure-code-plugins 2026-02-20T17:57:47.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:47.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-20T17:57:47.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:47.997 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T17:57:47.997 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-20T17:57:47.997 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T17:57:47.997 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T17:57:47.997 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T17:57:47.997 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T17:57:47.997 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T17:57:47.998 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T17:57:47.998 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T17:57:47.998 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T17:57:47.998 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T17:57:47.999 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T17:57:47.999 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T17:57:47.999 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T17:57:48.000 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:57:48.000 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T17:57:48.000 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T17:57:48.000 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:57:48.001 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T17:57:48.002 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T17:57:48.002 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-20T17:57:48.003 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T17:57:48.003 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:48.003 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:48.003 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.32971 2026-02-20T17:57:48.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T17:57:48.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T17:57:48.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-20T17:57:48.005 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-20T17:57:48.005 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:48.005 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:48.006 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.32971 2026-02-20T17:57:48.006 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-20T17:57:48.008 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-20T17:57:48.008 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-20T17:57:48.008 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-20T17:57:48.008 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:62: TEST_preload_no_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=isa 2026-02-20T17:57:48.008 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:48.008 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-20T17:57:48.008 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-20T17:57:48.008 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-20T17:57:48.008 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-20T17:57:48.008 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=isa 2026-02-20T17:57:48.027 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-20T17:57:48.027 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:57:48.027 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:57:48.028 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:57:48.028 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:48.028 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:48.029 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:48.029 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=isa 2026-02-20T17:57:48.052 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-20T17:57:48.053 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-20T17:57:48.053 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T17:57:48.053 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T17:57:48.053 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-20T17:57:48.054 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-20T17:57:48.054 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T17:57:48.054 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:57:48.054 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:57:48.055 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:48.055 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:48.055 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:48.055 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:57:48.055 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T17:57:48.055 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.32971/ceph-mon.a.asok config get fsid 2026-02-20T17:57:48.099 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-20T17:57:48.099 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T17:57:48.099 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T17:57:48.099 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-20T17:57:48.099 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-20T17:57:48.099 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T17:57:48.099 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:57:48.100 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:57:48.100 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:48.100 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:48.100 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:48.100 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:57:48.100 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T17:57:48.101 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.32971/ceph-mon.a.asok config get mon_host 2026-02-20T17:57:48.143 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:63: TEST_preload_no_warning: run_mgr td/test-erasure-code-plugins x 2026-02-20T17:57:48.143 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:48.143 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-20T17:57:48.143 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-20T17:57:48.144 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-20T17:57:48.144 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-20T17:57:48.144 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-20T17:57:48.241 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-20T17:57:48.241 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:57:48.241 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:57:48.241 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:57:48.241 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:48.241 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:48.241 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:48.242 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T17:57:48.242 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T17:57:48.259 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: get_asok_path mon.a 2026-02-20T17:57:48.260 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:57:48.260 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:57:48.260 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:48.260 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:48.260 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:48.260 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:57:48.260 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: CEPH_ARGS= 2026-02-20T17:57:48.260 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.32971/ceph-mon.a.asok log flush 2026-02-20T17:57:48.300 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T17:57:48.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:65: TEST_preload_no_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=isa 2026-02-20T17:57:48.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:48.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T17:57:48.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-20T17:57:48.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T17:57:48.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-20T17:57:48.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-20T17:57:48.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T17:57:48.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T17:57:48.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T17:57:48.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-20T17:57:48.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-20T17:57:48.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T17:57:48.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T17:57:48.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-20T17:57:48.309 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T17:57:48.309 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:57:48.309 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:57:48.309 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:57:48.309 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:48.309 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:48.309 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:48.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:48.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T17:57:48.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T17:57:48.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T17:57:48.310 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-20T17:57:48.310 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-20T17:57:48.310 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T17:57:48.310 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T17:57:48.310 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T17:57:48.310 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T17:57:48.310 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T17:57:48.310 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=isa 2026-02-20T17:57:48.310 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-20T17:57:48.310 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T17:57:48.311 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0e24afe9-ba0f-4b9f-abbd-3503bef97f8f 2026-02-20T17:57:48.311 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 0e24afe9-ba0f-4b9f-abbd-3503bef97f8f' 2026-02-20T17:57:48.311 INFO:tasks.workunit.client.0.trial195.stdout:add osd0 0e24afe9-ba0f-4b9f-abbd-3503bef97f8f 2026-02-20T17:57:48.312 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T17:57:48.321 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCcoJhpjGMAExAAspjSrM3Q/9DwIcfezDyrww== 2026-02-20T17:57:48.321 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCcoJhpjGMAExAAspjSrM3Q/9DwIcfezDyrww=="}' 2026-02-20T17:57:48.321 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0e24afe9-ba0f-4b9f-abbd-3503bef97f8f -i td/test-erasure-code-plugins/0/new.json 2026-02-20T17:57:48.421 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T17:57:48.427 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-20T17:57:48.428 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=isa --mkfs --key AQCcoJhpjGMAExAAspjSrM3Q/9DwIcfezDyrww== --osd-uuid 0e24afe9-ba0f-4b9f-abbd-3503bef97f8f 2026-02-20T17:57:48.445 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:48.443+0000 7f0367962900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:48.446 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:48.445+0000 7f0367962900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:48.447 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:48.446+0000 7f0367962900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:48.448 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:48.446+0000 7f0367962900 -1 bdev(0x558cd39d8c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-20T17:57:48.448 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:48.446+0000 7f0367962900 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-20T17:57:48.847 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-20T17:57:48.847 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T17:57:48.847 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-20T17:57:48.848 INFO:tasks.workunit.client.0.trial195.stdout:adding osd0 key to auth repository 2026-02-20T17:57:48.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T17:57:48.949 INFO:tasks.workunit.client.0.trial195.stdout:start osd.0 2026-02-20T17:57:48.950 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-20T17:57:48.950 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=isa 2026-02-20T17:57:48.950 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T17:57:48.950 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T17:57:48.950 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T17:57:48.966 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:48.965+0000 7f9b095ee900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:48.967 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:48.966+0000 7f9b095ee900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:48.968 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:48.967+0000 7f9b095ee900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:49.047 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-20T17:57:49.048 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T17:57:49.048 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-20T17:57:49.048 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T17:57:49.048 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T17:57:49.048 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:57:49.048 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T17:57:49.048 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T17:57:49.049 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:57:49.049 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T17:57:49.144 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T17:57:49.169 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:49.168+0000 7f9b095ee900 -1 Falling back to public interface 2026-02-20T17:57:49.482 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:49.480+0000 7f9b095ee900 -1 osd.0 0 log_to_monitors true 2026-02-20T17:57:50.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T17:57:50.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:57:50.146 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T17:57:50.146 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T17:57:50.146 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:57:50.146 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T17:57:50.244 INFO:tasks.workunit.client.0.trial195.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2746307519,v1:127.0.0.1:6801/2746307519] [v2:127.0.0.1:6802/2746307519,v1:127.0.0.1:6803/2746307519] exists,up 0e24afe9-ba0f-4b9f-abbd-3503bef97f8f 2026-02-20T17:57:50.246 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T17:57:50.246 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T17:57:50.246 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T17:57:50.246 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: get_asok_path osd.0 2026-02-20T17:57:50.246 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-20T17:57:50.246 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-20T17:57:50.247 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:50.247 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:50.247 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:50.247 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-osd.0.asok 2026-02-20T17:57:50.247 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: CEPH_ARGS= 2026-02-20T17:57:50.247 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.32971/ceph-osd.0.asok log flush 2026-02-20T17:57:50.295 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T17:57:50.300 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:67: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/mon.a.log 2026-02-20T17:57:50.301 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:68: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/osd.0.log 2026-02-20T17:57:50.303 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:69: TEST_preload_no_warning: teardown td/test-erasure-code-plugins 2026-02-20T17:57:50.304 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:50.304 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T17:57:50.304 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-20T17:57:50.304 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T17:57:50.304 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T17:57:50.304 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T17:57:50.304 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T17:57:50.304 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T17:57:50.410 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T17:57:50.411 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T17:57:50.411 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T17:57:50.412 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T17:57:50.413 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T17:57:50.413 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T17:57:50.413 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T17:57:50.414 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:57:50.414 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T17:57:50.415 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T17:57:50.415 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:57:50.416 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T17:57:50.417 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T17:57:50.417 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-20T17:57:50.420 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T17:57:50.420 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:50.420 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:50.420 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.32971 2026-02-20T17:57:50.422 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T17:57:50.422 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T17:57:50.422 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:72: TEST_preload_no_warning: return 0 2026-02-20T17:57:50.423 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-20T17:57:50.423 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_preload_no_warning_default td/test-erasure-code-plugins 2026-02-20T17:57:50.423 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:76: TEST_preload_no_warning_default: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:50.423 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:78: TEST_preload_no_warning_default: setup td/test-erasure-code-plugins 2026-02-20T17:57:50.423 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:50.423 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-20T17:57:50.423 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:50.423 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T17:57:50.423 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-20T17:57:50.423 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T17:57:50.423 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T17:57:50.424 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T17:57:50.424 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T17:57:50.424 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T17:57:50.425 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T17:57:50.425 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T17:57:50.425 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T17:57:50.425 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T17:57:50.425 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T17:57:50.425 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T17:57:50.426 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T17:57:50.426 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:57:50.426 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T17:57:50.427 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T17:57:50.427 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:57:50.427 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T17:57:50.428 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T17:57:50.428 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-20T17:57:50.429 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T17:57:50.429 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:50.429 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:50.429 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.32971 2026-02-20T17:57:50.430 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T17:57:50.430 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T17:57:50.430 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-20T17:57:50.431 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-20T17:57:50.431 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:50.431 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:50.431 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.32971 2026-02-20T17:57:50.432 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-20T17:57:50.433 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-20T17:57:50.434 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-20T17:57:50.434 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-20T17:57:50.434 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:79: TEST_preload_no_warning_default: run_mon td/test-erasure-code-plugins a 2026-02-20T17:57:50.434 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:50.434 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-20T17:57:50.434 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-20T17:57:50.434 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-20T17:57:50.434 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-20T17:57:50.434 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins 2026-02-20T17:57:50.452 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-20T17:57:50.452 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:57:50.453 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:57:50.453 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:57:50.453 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:50.453 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:50.453 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:50.453 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-20T17:57:50.478 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-20T17:57:50.480 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-20T17:57:50.480 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T17:57:50.480 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T17:57:50.480 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-20T17:57:50.480 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-20T17:57:50.481 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T17:57:50.481 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:57:50.481 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:57:50.481 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:50.481 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:50.482 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:50.482 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:57:50.482 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T17:57:50.482 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.32971/ceph-mon.a.asok config get fsid 2026-02-20T17:57:50.525 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-20T17:57:50.525 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T17:57:50.525 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T17:57:50.525 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-20T17:57:50.526 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-20T17:57:50.527 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T17:57:50.527 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:57:50.527 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:57:50.527 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:50.527 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:50.527 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:50.527 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:57:50.527 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T17:57:50.527 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.32971/ceph-mon.a.asok config get mon_host 2026-02-20T17:57:50.570 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:80: TEST_preload_no_warning_default: get_asok_path mon.a 2026-02-20T17:57:50.570 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:57:50.570 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:57:50.571 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:50.571 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:50.571 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:50.571 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:57:50.571 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:80: TEST_preload_no_warning_default: CEPH_ARGS= 2026-02-20T17:57:50.571 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:80: TEST_preload_no_warning_default: ceph --admin-daemon /tmp/ceph-asok.32971/ceph-mon.a.asok log flush 2026-02-20T17:57:50.609 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T17:57:50.613 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:81: TEST_preload_no_warning_default: run_mgr td/test-erasure-code-plugins x 2026-02-20T17:57:50.613 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:50.613 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-20T17:57:50.613 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-20T17:57:50.613 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-20T17:57:50.613 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-20T17:57:50.613 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-20T17:57:50.712 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-20T17:57:50.712 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:57:50.712 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:57:50.712 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:57:50.712 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:50.712 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:50.713 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:50.713 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T17:57:50.714 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T17:57:50.731 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:82: TEST_preload_no_warning_default: run_osd td/test-erasure-code-plugins 0 2026-02-20T17:57:50.731 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:50.731 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T17:57:50.731 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-20T17:57:50.731 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T17:57:50.731 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-20T17:57:50.731 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-20T17:57:50.731 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T17:57:50.731 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T17:57:50.731 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T17:57:50.731 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-20T17:57:50.731 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-20T17:57:50.731 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T17:57:50.731 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T17:57:50.731 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-20T17:57:50.731 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T17:57:50.731 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:57:50.731 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:57:50.731 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:57:50.731 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:50.731 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:50.732 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:50.732 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:50.732 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T17:57:50.732 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T17:57:50.732 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T17:57:50.732 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-20T17:57:50.732 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-20T17:57:50.732 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T17:57:50.732 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T17:57:50.732 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T17:57:50.732 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T17:57:50.732 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T17:57:50.732 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T17:57:50.732 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-20T17:57:50.733 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T17:57:50.734 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2fdffea4-d6cf-4811-9133-210b255b9c76 2026-02-20T17:57:50.734 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 2fdffea4-d6cf-4811-9133-210b255b9c76' 2026-02-20T17:57:50.734 INFO:tasks.workunit.client.0.trial195.stdout:add osd0 2fdffea4-d6cf-4811-9133-210b255b9c76 2026-02-20T17:57:50.734 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T17:57:50.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCeoJhpFw9NLBAAvsRXRegW1hkNPNILtbux6w== 2026-02-20T17:57:50.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCeoJhpFw9NLBAAvsRXRegW1hkNPNILtbux6w=="}' 2026-02-20T17:57:50.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2fdffea4-d6cf-4811-9133-210b255b9c76 -i td/test-erasure-code-plugins/0/new.json 2026-02-20T17:57:50.847 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T17:57:50.853 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-20T17:57:50.854 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCeoJhpFw9NLBAAvsRXRegW1hkNPNILtbux6w== --osd-uuid 2fdffea4-d6cf-4811-9133-210b255b9c76 2026-02-20T17:57:50.871 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:50.870+0000 7fab1f922900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:50.872 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:50.871+0000 7fab1f922900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:50.873 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:50.872+0000 7fab1f922900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:50.874 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:50.873+0000 7fab1f922900 -1 bdev(0x55825032ec00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-20T17:57:50.874 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:50.873+0000 7fab1f922900 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-20T17:57:51.297 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-20T17:57:51.297 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T17:57:51.298 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-20T17:57:51.298 INFO:tasks.workunit.client.0.trial195.stdout:adding osd0 key to auth repository 2026-02-20T17:57:51.298 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T17:57:51.401 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-20T17:57:51.401 INFO:tasks.workunit.client.0.trial195.stdout:start osd.0 2026-02-20T17:57:51.401 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T17:57:51.401 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T17:57:51.401 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T17:57:51.401 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T17:57:51.417 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:51.416+0000 7f666f542900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:51.419 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:51.418+0000 7f666f542900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:51.420 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:51.419+0000 7f666f542900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:51.498 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-20T17:57:51.498 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T17:57:51.498 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-20T17:57:51.498 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T17:57:51.498 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T17:57:51.499 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:57:51.499 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T17:57:51.499 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T17:57:51.499 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:57:51.499 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T17:57:51.596 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T17:57:51.598 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:51.598+0000 7f666f542900 -1 Falling back to public interface 2026-02-20T17:57:51.981 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:51.980+0000 7f666f542900 -1 osd.0 0 log_to_monitors true 2026-02-20T17:57:52.597 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T17:57:52.597 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:57:52.597 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T17:57:52.597 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T17:57:52.597 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:57:52.597 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T17:57:52.696 INFO:tasks.workunit.client.0.trial195.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3775464378,v1:127.0.0.1:6801/3775464378] [v2:127.0.0.1:6802/3775464378,v1:127.0.0.1:6803/3775464378] exists,up 2fdffea4-d6cf-4811-9133-210b255b9c76 2026-02-20T17:57:52.696 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T17:57:52.696 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T17:57:52.696 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T17:57:52.696 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:83: TEST_preload_no_warning_default: get_asok_path osd.0 2026-02-20T17:57:52.696 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-20T17:57:52.696 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-20T17:57:52.697 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:52.697 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:52.697 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:52.697 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-osd.0.asok 2026-02-20T17:57:52.697 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:83: TEST_preload_no_warning_default: CEPH_ARGS= 2026-02-20T17:57:52.697 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:83: TEST_preload_no_warning_default: ceph --admin-daemon /tmp/ceph-asok.32971/ceph-osd.0.asok log flush 2026-02-20T17:57:52.745 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T17:57:52.749 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:84: TEST_preload_no_warning_default: grep 'WARNING: osd_erasure_code_plugins' td/test-erasure-code-plugins/mon.a.log 2026-02-20T17:57:52.750 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:85: TEST_preload_no_warning_default: grep 'WARNING: osd_erasure_code_plugins' td/test-erasure-code-plugins/osd.0.log 2026-02-20T17:57:52.752 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:86: TEST_preload_no_warning_default: teardown td/test-erasure-code-plugins 2026-02-20T17:57:52.753 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:52.753 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T17:57:52.753 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-20T17:57:52.753 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T17:57:52.753 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T17:57:52.753 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T17:57:52.753 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T17:57:52.753 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T17:57:52.859 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T17:57:52.859 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T17:57:52.860 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T17:57:52.860 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T17:57:52.861 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T17:57:52.862 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T17:57:52.862 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T17:57:52.863 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:57:52.863 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T17:57:52.863 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T17:57:52.864 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:57:52.864 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T17:57:52.865 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T17:57:52.865 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-20T17:57:52.868 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T17:57:52.868 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:52.868 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:52.868 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.32971 2026-02-20T17:57:52.871 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T17:57:52.871 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T17:57:52.872 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:88: TEST_preload_no_warning_default: return 0 2026-02-20T17:57:52.872 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-20T17:57:52.872 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_preload_warning td/test-erasure-code-plugins 2026-02-20T17:57:52.872 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:41: TEST_preload_warning: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:52.872 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-20T17:57:52.872 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-20T17:57:52.872 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:52.872 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-20T17:57:52.872 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:52.872 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T17:57:52.872 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-20T17:57:52.872 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T17:57:52.873 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T17:57:52.873 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T17:57:52.873 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T17:57:52.873 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T17:57:52.874 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T17:57:52.874 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T17:57:52.874 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T17:57:52.875 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T17:57:52.875 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T17:57:52.875 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T17:57:52.875 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T17:57:52.876 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:57:52.876 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T17:57:52.876 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T17:57:52.876 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:57:52.876 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T17:57:52.877 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T17:57:52.877 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-20T17:57:52.878 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T17:57:52.878 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:52.878 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:52.878 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.32971 2026-02-20T17:57:52.879 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T17:57:52.879 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T17:57:52.879 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-20T17:57:52.880 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-20T17:57:52.881 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:52.881 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:52.881 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.32971 2026-02-20T17:57:52.883 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-20T17:57:52.883 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-20T17:57:52.883 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-20T17:57:52.883 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-20T17:57:52.883 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=jerasure_generic 2026-02-20T17:57:52.883 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:52.883 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-20T17:57:52.884 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-20T17:57:52.884 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-20T17:57:52.884 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-20T17:57:52.884 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=jerasure_generic 2026-02-20T17:57:52.903 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-20T17:57:52.904 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:57:52.904 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:57:52.904 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:57:52.904 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:52.904 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:52.904 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:52.904 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=jerasure_generic 2026-02-20T17:57:52.929 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-20T17:57:52.931 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-20T17:57:52.931 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T17:57:52.931 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T17:57:52.931 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-20T17:57:52.931 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-20T17:57:52.932 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T17:57:52.932 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:57:52.932 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:57:52.932 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:52.932 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:52.932 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:52.932 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:57:52.932 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T17:57:52.932 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.32971/ceph-mon.a.asok config get fsid 2026-02-20T17:57:52.976 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-20T17:57:52.976 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T17:57:52.976 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T17:57:52.976 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-20T17:57:52.976 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-20T17:57:52.976 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T17:57:52.976 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:57:52.977 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:57:52.977 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:52.977 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:52.977 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:52.977 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:57:52.977 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T17:57:52.977 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.32971/ceph-mon.a.asok config get mon_host 2026-02-20T17:57:53.021 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-20T17:57:53.021 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:53.021 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-20T17:57:53.022 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-20T17:57:53.022 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-20T17:57:53.022 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-20T17:57:53.022 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-20T17:57:53.124 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-20T17:57:53.124 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:57:53.124 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:57:53.124 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:57:53.125 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:53.125 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:53.125 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:53.125 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T17:57:53.125 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T17:57:53.143 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-20T17:57:53.143 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:57:53.143 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:57:53.143 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:53.143 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:53.143 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:53.143 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:57:53.144 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-20T17:57:53.144 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.32971/ceph-mon.a.asok log flush 2026-02-20T17:57:53.184 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T17:57:53.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=jerasure_generic 2026-02-20T17:57:53.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:53.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T17:57:53.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-20T17:57:53.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T17:57:53.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-20T17:57:53.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-20T17:57:53.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T17:57:53.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T17:57:53.192 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T17:57:53.192 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-20T17:57:53.192 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-20T17:57:53.192 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T17:57:53.192 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T17:57:53.192 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-20T17:57:53.192 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T17:57:53.192 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:57:53.192 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:57:53.192 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:57:53.192 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:53.192 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:53.192 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:53.193 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:53.193 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T17:57:53.193 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T17:57:53.193 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T17:57:53.193 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-20T17:57:53.193 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-20T17:57:53.193 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T17:57:53.193 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T17:57:53.193 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T17:57:53.193 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T17:57:53.193 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T17:57:53.193 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=jerasure_generic 2026-02-20T17:57:53.194 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-20T17:57:53.194 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T17:57:53.194 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ccb5cf94-837c-4c75-b1e6-9f211a7489e1 2026-02-20T17:57:53.194 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 ccb5cf94-837c-4c75-b1e6-9f211a7489e1' 2026-02-20T17:57:53.195 INFO:tasks.workunit.client.0.trial195.stdout:add osd0 ccb5cf94-837c-4c75-b1e6-9f211a7489e1 2026-02-20T17:57:53.195 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T17:57:53.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQChoJhpfqsLDBAAv0n4jd/UYgyUZ8zXxSYRUg== 2026-02-20T17:57:53.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQChoJhpfqsLDBAAv0n4jd/UYgyUZ8zXxSYRUg=="}' 2026-02-20T17:57:53.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ccb5cf94-837c-4c75-b1e6-9f211a7489e1 -i td/test-erasure-code-plugins/0/new.json 2026-02-20T17:57:53.308 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T17:57:53.314 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-20T17:57:53.315 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_generic --mkfs --key AQChoJhpfqsLDBAAv0n4jd/UYgyUZ8zXxSYRUg== --osd-uuid ccb5cf94-837c-4c75-b1e6-9f211a7489e1 2026-02-20T17:57:53.333 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:53.331+0000 7fb4d2015900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:53.334 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:53.333+0000 7fb4d2015900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:53.335 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:53.333+0000 7fb4d2015900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:53.335 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:53.334+0000 7fb4d2015900 -1 bdev(0x559ad575ac00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-20T17:57:53.336 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:53.334+0000 7fb4d2015900 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-20T17:57:53.738 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-20T17:57:53.739 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T17:57:53.739 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-20T17:57:53.739 INFO:tasks.workunit.client.0.trial195.stdout:adding osd0 key to auth repository 2026-02-20T17:57:53.740 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T17:57:53.845 INFO:tasks.workunit.client.0.trial195.stdout:start osd.0 2026-02-20T17:57:53.845 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-20T17:57:53.845 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_generic 2026-02-20T17:57:53.845 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T17:57:53.845 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T17:57:53.845 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T17:57:53.861 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:53.860+0000 7fc33a40d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:53.862 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:53.861+0000 7fc33a40d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:53.863 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:53.862+0000 7fc33a40d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:53.947 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-20T17:57:53.947 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T17:57:53.948 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-20T17:57:53.948 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T17:57:53.948 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T17:57:53.948 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:57:53.948 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T17:57:53.948 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T17:57:53.949 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:57:53.949 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T17:57:54.043 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:54.041+0000 7fc33a40d900 -1 Falling back to public interface 2026-02-20T17:57:54.044 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T17:57:54.402 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:54.401+0000 7fc33a40d900 -1 osd.0 0 log_to_monitors true 2026-02-20T17:57:55.046 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T17:57:55.046 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:57:55.046 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T17:57:55.047 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T17:57:55.047 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:57:55.047 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T17:57:55.146 INFO:tasks.workunit.client.0.trial195.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/912631001,v1:127.0.0.1:6801/912631001] [v2:127.0.0.1:6802/912631001,v1:127.0.0.1:6803/912631001] exists,up ccb5cf94-837c-4c75-b1e6-9f211a7489e1 2026-02-20T17:57:55.147 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T17:57:55.148 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T17:57:55.148 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T17:57:55.148 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-20T17:57:55.148 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-20T17:57:55.148 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-20T17:57:55.148 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:55.148 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:55.148 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:55.148 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-osd.0.asok 2026-02-20T17:57:55.148 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-20T17:57:55.148 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.32971/ceph-osd.0.asok log flush 2026-02-20T17:57:55.192 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T17:57:55.196 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_generic' td/test-erasure-code-plugins/mon.a.log 2026-02-20T17:57:55.197 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T17:57:52.919+0000 7f2503ff9d80 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_generic that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-20T17:57:55.198 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_generic' td/test-erasure-code-plugins/osd.0.log 2026-02-20T17:57:55.198 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T17:57:54.042+0000 7fc33a40d900 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_generic that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-20T17:57:55.199 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-20T17:57:55.199 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:55.199 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T17:57:55.199 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-20T17:57:55.199 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T17:57:55.199 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T17:57:55.199 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T17:57:55.199 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T17:57:55.199 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T17:57:55.306 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T17:57:55.307 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T17:57:55.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T17:57:55.308 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T17:57:55.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T17:57:55.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T17:57:55.309 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T17:57:55.310 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:57:55.310 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T17:57:55.311 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T17:57:55.311 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:57:55.312 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T17:57:55.313 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T17:57:55.313 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-20T17:57:55.316 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T17:57:55.316 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:55.316 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:55.316 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.32971 2026-02-20T17:57:55.318 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T17:57:55.318 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T17:57:55.318 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-20T17:57:55.318 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-20T17:57:55.318 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:55.319 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-20T17:57:55.319 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:55.319 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T17:57:55.319 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-20T17:57:55.319 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T17:57:55.319 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T17:57:55.319 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T17:57:55.319 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T17:57:55.319 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T17:57:55.320 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T17:57:55.320 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T17:57:55.320 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T17:57:55.320 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T17:57:55.321 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T17:57:55.321 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T17:57:55.321 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T17:57:55.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:57:55.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T17:57:55.323 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T17:57:55.323 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:57:55.323 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T17:57:55.324 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T17:57:55.324 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-20T17:57:55.325 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T17:57:55.325 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:55.326 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:55.326 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.32971 2026-02-20T17:57:55.326 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T17:57:55.326 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T17:57:55.327 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-20T17:57:55.327 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-20T17:57:55.328 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:55.328 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:55.328 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.32971 2026-02-20T17:57:55.329 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-20T17:57:55.330 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-20T17:57:55.330 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-20T17:57:55.330 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-20T17:57:55.330 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=jerasure_sse3 2026-02-20T17:57:55.330 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:55.330 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-20T17:57:55.330 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-20T17:57:55.331 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-20T17:57:55.331 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-20T17:57:55.331 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=jerasure_sse3 2026-02-20T17:57:55.349 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-20T17:57:55.349 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:57:55.349 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:57:55.350 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:57:55.350 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:55.350 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:55.350 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:55.350 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=jerasure_sse3 2026-02-20T17:57:55.375 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-20T17:57:55.377 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-20T17:57:55.377 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T17:57:55.378 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T17:57:55.378 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-20T17:57:55.378 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-20T17:57:55.378 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T17:57:55.378 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:57:55.378 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:57:55.378 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:55.378 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:55.378 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:55.378 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:57:55.378 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T17:57:55.379 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.32971/ceph-mon.a.asok config get fsid 2026-02-20T17:57:55.423 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-20T17:57:55.424 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T17:57:55.424 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T17:57:55.424 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-20T17:57:55.424 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-20T17:57:55.424 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T17:57:55.424 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:57:55.424 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:57:55.424 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:55.424 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:55.424 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:55.424 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:57:55.425 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T17:57:55.425 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.32971/ceph-mon.a.asok config get mon_host 2026-02-20T17:57:55.469 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-20T17:57:55.469 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:55.469 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-20T17:57:55.469 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-20T17:57:55.469 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-20T17:57:55.469 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-20T17:57:55.469 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-20T17:57:55.569 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-20T17:57:55.570 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:57:55.570 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:57:55.570 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:57:55.570 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:55.570 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:55.570 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:55.571 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T17:57:55.571 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T17:57:55.589 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-20T17:57:55.589 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:57:55.589 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:57:55.589 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:55.590 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:55.590 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:55.590 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:57:55.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-20T17:57:55.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.32971/ceph-mon.a.asok log flush 2026-02-20T17:57:55.632 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T17:57:55.640 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=jerasure_sse3 2026-02-20T17:57:55.641 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:55.641 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T17:57:55.641 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-20T17:57:55.641 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T17:57:55.641 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-20T17:57:55.641 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-20T17:57:55.641 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T17:57:55.641 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T17:57:55.641 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T17:57:55.642 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-20T17:57:55.642 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-20T17:57:55.642 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T17:57:55.642 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T17:57:55.642 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-20T17:57:55.642 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T17:57:55.642 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:57:55.642 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:57:55.642 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:57:55.642 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:55.642 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:55.642 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:55.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:55.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T17:57:55.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T17:57:55.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T17:57:55.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-20T17:57:55.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-20T17:57:55.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T17:57:55.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T17:57:55.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T17:57:55.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T17:57:55.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T17:57:55.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=jerasure_sse3 2026-02-20T17:57:55.644 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-20T17:57:55.644 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T17:57:55.644 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=6d29e565-1800-4389-b9a9-df58f4629294 2026-02-20T17:57:55.644 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 6d29e565-1800-4389-b9a9-df58f4629294' 2026-02-20T17:57:55.644 INFO:tasks.workunit.client.0.trial195.stdout:add osd0 6d29e565-1800-4389-b9a9-df58f4629294 2026-02-20T17:57:55.645 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T17:57:55.650 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCjoJhpYNetJhAATFtFKljqAxu3Sf9WYOozBQ== 2026-02-20T17:57:55.650 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCjoJhpYNetJhAATFtFKljqAxu3Sf9WYOozBQ=="}' 2026-02-20T17:57:55.650 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 6d29e565-1800-4389-b9a9-df58f4629294 -i td/test-erasure-code-plugins/0/new.json 2026-02-20T17:57:55.752 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T17:57:55.759 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-20T17:57:55.760 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_sse3 --mkfs --key AQCjoJhpYNetJhAATFtFKljqAxu3Sf9WYOozBQ== --osd-uuid 6d29e565-1800-4389-b9a9-df58f4629294 2026-02-20T17:57:55.777 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:55.776+0000 7f39e7cf3900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:55.778 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:55.777+0000 7f39e7cf3900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:55.779 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:55.778+0000 7f39e7cf3900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:55.780 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:55.779+0000 7f39e7cf3900 -1 bdev(0x55dfbe792c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-20T17:57:55.780 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:55.779+0000 7f39e7cf3900 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-20T17:57:56.169 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-20T17:57:56.170 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T17:57:56.170 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-20T17:57:56.171 INFO:tasks.workunit.client.0.trial195.stdout:adding osd0 key to auth repository 2026-02-20T17:57:56.171 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T17:57:56.276 INFO:tasks.workunit.client.0.trial195.stdout:start osd.0 2026-02-20T17:57:56.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-20T17:57:56.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_sse3 2026-02-20T17:57:56.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T17:57:56.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T17:57:56.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T17:57:56.292 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:56.291+0000 7fd0ccc11900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:56.294 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:56.293+0000 7fd0ccc11900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:56.295 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:56.293+0000 7fd0ccc11900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:56.375 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-20T17:57:56.375 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T17:57:56.375 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-20T17:57:56.375 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T17:57:56.375 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T17:57:56.375 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:57:56.375 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T17:57:56.376 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T17:57:56.376 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:57:56.376 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T17:57:56.472 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T17:57:56.496 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:56.494+0000 7fd0ccc11900 -1 Falling back to public interface 2026-02-20T17:57:56.877 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:56.876+0000 7fd0ccc11900 -1 osd.0 0 log_to_monitors true 2026-02-20T17:57:57.474 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T17:57:57.474 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:57:57.474 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T17:57:57.475 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T17:57:57.475 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:57:57.475 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T17:57:57.572 INFO:tasks.workunit.client.0.trial195.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3468462224,v1:127.0.0.1:6801/3468462224] [v2:127.0.0.1:6802/3468462224,v1:127.0.0.1:6803/3468462224] exists,up 6d29e565-1800-4389-b9a9-df58f4629294 2026-02-20T17:57:57.573 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T17:57:57.573 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T17:57:57.573 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T17:57:57.573 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-20T17:57:57.573 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-20T17:57:57.573 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-20T17:57:57.573 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:57.573 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:57.573 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:57.574 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-osd.0.asok 2026-02-20T17:57:57.575 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-20T17:57:57.575 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.32971/ceph-osd.0.asok log flush 2026-02-20T17:57:57.619 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T17:57:57.622 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_sse3' td/test-erasure-code-plugins/mon.a.log 2026-02-20T17:57:57.624 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T17:57:55.365+0000 7f2b73ddad80 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_sse3 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-20T17:57:57.624 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_sse3' td/test-erasure-code-plugins/osd.0.log 2026-02-20T17:57:57.625 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T17:57:56.495+0000 7fd0ccc11900 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_sse3 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-20T17:57:57.626 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-20T17:57:57.626 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:57.626 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T17:57:57.626 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-20T17:57:57.626 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T17:57:57.626 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T17:57:57.627 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T17:57:57.627 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T17:57:57.627 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T17:57:57.732 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T17:57:57.733 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T17:57:57.734 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T17:57:57.734 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T17:57:57.735 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T17:57:57.735 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T17:57:57.736 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T17:57:57.737 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:57:57.737 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T17:57:57.737 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T17:57:57.737 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:57:57.738 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T17:57:57.739 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T17:57:57.739 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-20T17:57:57.742 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T17:57:57.742 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:57.742 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:57.742 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.32971 2026-02-20T17:57:57.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T17:57:57.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T17:57:57.745 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-20T17:57:57.745 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-20T17:57:57.745 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:57.745 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-20T17:57:57.745 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:57.745 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T17:57:57.745 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-20T17:57:57.745 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T17:57:57.745 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T17:57:57.745 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T17:57:57.745 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T17:57:57.745 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T17:57:57.746 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T17:57:57.746 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T17:57:57.746 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T17:57:57.747 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T17:57:57.747 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T17:57:57.747 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T17:57:57.748 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T17:57:57.748 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:57:57.748 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T17:57:57.749 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T17:57:57.749 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:57:57.750 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T17:57:57.750 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T17:57:57.751 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-20T17:57:57.752 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T17:57:57.752 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:57.752 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:57.752 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.32971 2026-02-20T17:57:57.753 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T17:57:57.753 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T17:57:57.753 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-20T17:57:57.754 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-20T17:57:57.754 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:57.754 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:57.754 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.32971 2026-02-20T17:57:57.756 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-20T17:57:57.756 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-20T17:57:57.757 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-20T17:57:57.757 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-20T17:57:57.757 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=jerasure_sse4 2026-02-20T17:57:57.757 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:57.757 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-20T17:57:57.757 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-20T17:57:57.757 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-20T17:57:57.757 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-20T17:57:57.757 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=jerasure_sse4 2026-02-20T17:57:57.776 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-20T17:57:57.776 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:57:57.776 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:57:57.776 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:57:57.776 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:57.777 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:57.777 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:57.777 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=jerasure_sse4 2026-02-20T17:57:57.802 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-20T17:57:57.804 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-20T17:57:57.804 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T17:57:57.804 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T17:57:57.804 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-20T17:57:57.804 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-20T17:57:57.804 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T17:57:57.805 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:57:57.805 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:57:57.805 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:57.805 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:57.805 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:57.805 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:57:57.805 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T17:57:57.805 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.32971/ceph-mon.a.asok config get fsid 2026-02-20T17:57:57.851 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-20T17:57:57.851 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T17:57:57.851 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T17:57:57.851 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-20T17:57:57.851 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-20T17:57:57.852 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T17:57:57.852 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:57:57.852 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:57:57.852 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:57.852 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:57.852 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:57.852 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:57:57.853 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T17:57:57.853 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.32971/ceph-mon.a.asok config get mon_host 2026-02-20T17:57:57.895 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-20T17:57:57.896 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:57.896 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-20T17:57:57.896 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-20T17:57:57.896 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-20T17:57:57.896 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-20T17:57:57.896 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-20T17:57:57.997 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-20T17:57:57.997 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:57:57.997 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:57:57.997 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:57:57.997 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:57.997 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:57.997 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:57.998 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T17:57:57.998 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T17:57:58.016 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-20T17:57:58.017 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:57:58.017 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:57:58.017 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:57:58.017 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:58.017 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:58.017 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:57:58.017 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-20T17:57:58.017 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.32971/ceph-mon.a.asok log flush 2026-02-20T17:57:58.056 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T17:57:58.062 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=jerasure_sse4 2026-02-20T17:57:58.063 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-20T17:57:58.063 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T17:57:58.063 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-20T17:57:58.063 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T17:57:58.063 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-20T17:57:58.063 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-20T17:57:58.063 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T17:57:58.063 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T17:57:58.063 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T17:57:58.063 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-20T17:57:58.064 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-20T17:57:58.064 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T17:57:58.064 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T17:57:58.064 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-20T17:57:58.064 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T17:57:58.064 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:57:58.064 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:57:58.064 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:57:58.064 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:57:58.064 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:57:58.064 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:58.064 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:57:58.065 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T17:57:58.065 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T17:57:58.065 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T17:57:58.065 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-20T17:57:58.065 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-20T17:57:58.065 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T17:57:58.065 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T17:57:58.065 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T17:57:58.065 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T17:57:58.065 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T17:57:58.065 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=jerasure_sse4 2026-02-20T17:57:58.065 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-20T17:57:58.066 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T17:57:58.066 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8cbbaaec-8606-4ffe-8844-ee32adaf805a 2026-02-20T17:57:58.066 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 8cbbaaec-8606-4ffe-8844-ee32adaf805a' 2026-02-20T17:57:58.067 INFO:tasks.workunit.client.0.trial195.stdout:add osd0 8cbbaaec-8606-4ffe-8844-ee32adaf805a 2026-02-20T17:57:58.067 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T17:57:58.074 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCmoJhp4EVbBBAA4SvxLohIXzY1TrIaDMmusA== 2026-02-20T17:57:58.074 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCmoJhp4EVbBBAA4SvxLohIXzY1TrIaDMmusA=="}' 2026-02-20T17:57:58.075 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8cbbaaec-8606-4ffe-8844-ee32adaf805a -i td/test-erasure-code-plugins/0/new.json 2026-02-20T17:57:58.179 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T17:57:58.186 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-20T17:57:58.187 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_sse4 --mkfs --key AQCmoJhp4EVbBBAA4SvxLohIXzY1TrIaDMmusA== --osd-uuid 8cbbaaec-8606-4ffe-8844-ee32adaf805a 2026-02-20T17:57:58.204 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:58.203+0000 7fe1e9893900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:58.206 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:58.204+0000 7fe1e9893900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:58.207 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:58.205+0000 7fe1e9893900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:58.207 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:58.206+0000 7fe1e9893900 -1 bdev(0x5624d4f8cc00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-20T17:57:58.207 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:58.206+0000 7fe1e9893900 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-20T17:57:58.603 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-20T17:57:58.603 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T17:57:58.604 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-20T17:57:58.604 INFO:tasks.workunit.client.0.trial195.stdout:adding osd0 key to auth repository 2026-02-20T17:57:58.605 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T17:57:58.708 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-20T17:57:58.708 INFO:tasks.workunit.client.0.trial195.stdout:start osd.0 2026-02-20T17:57:58.709 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_sse4 2026-02-20T17:57:58.709 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T17:57:58.709 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T17:57:58.709 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T17:57:58.724 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:58.723+0000 7f4e77764900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:58.726 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:58.724+0000 7f4e77764900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:58.727 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:58.725+0000 7f4e77764900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:57:58.807 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-20T17:57:58.807 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T17:57:58.808 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-20T17:57:58.808 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T17:57:58.808 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T17:57:58.808 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:57:58.808 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T17:57:58.809 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T17:57:58.810 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:57:58.810 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T17:57:58.903 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T17:57:58.924 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:58.923+0000 7f4e77764900 -1 Falling back to public interface 2026-02-20T17:57:59.284 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:57:59.282+0000 7f4e77764900 -1 osd.0 0 log_to_monitors true 2026-02-20T17:57:59.904 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T17:57:59.904 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:57:59.905 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T17:57:59.905 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T17:57:59.905 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:57:59.905 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T17:58:00.005 INFO:tasks.workunit.client.0.trial195.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2705701990,v1:127.0.0.1:6801/2705701990] [v2:127.0.0.1:6802/2705701990,v1:127.0.0.1:6803/2705701990] exists,up 8cbbaaec-8606-4ffe-8844-ee32adaf805a 2026-02-20T17:58:00.006 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T17:58:00.007 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T17:58:00.007 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T17:58:00.007 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-20T17:58:00.007 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-20T17:58:00.007 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-20T17:58:00.007 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:58:00.007 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:00.007 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:58:00.007 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-osd.0.asok 2026-02-20T17:58:00.007 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-20T17:58:00.007 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.32971/ceph-osd.0.asok log flush 2026-02-20T17:58:00.053 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T17:58:00.057 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_sse4' td/test-erasure-code-plugins/mon.a.log 2026-02-20T17:58:00.059 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T17:57:57.792+0000 7fdf01c0dd80 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_sse4 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-20T17:58:00.059 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_sse4' td/test-erasure-code-plugins/osd.0.log 2026-02-20T17:58:00.060 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T17:57:58.923+0000 7f4e77764900 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_sse4 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-20T17:58:00.061 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-20T17:58:00.061 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-20T17:58:00.062 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T17:58:00.062 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-20T17:58:00.062 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T17:58:00.062 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T17:58:00.062 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T17:58:00.062 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T17:58:00.062 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T17:58:00.168 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T17:58:00.169 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T17:58:00.169 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T17:58:00.170 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T17:58:00.171 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T17:58:00.171 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T17:58:00.171 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T17:58:00.172 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:58:00.173 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T17:58:00.173 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T17:58:00.173 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:58:00.174 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T17:58:00.174 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T17:58:00.175 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-20T17:58:00.177 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T17:58:00.178 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:00.178 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:58:00.178 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.32971 2026-02-20T17:58:00.179 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T17:58:00.180 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T17:58:00.180 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-20T17:58:00.180 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-20T17:58:00.180 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-20T17:58:00.180 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-20T17:58:00.180 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-20T17:58:00.180 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T17:58:00.180 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-20T17:58:00.180 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T17:58:00.180 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T17:58:00.180 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T17:58:00.181 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T17:58:00.181 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T17:58:00.181 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T17:58:00.181 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T17:58:00.182 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T17:58:00.182 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T17:58:00.183 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T17:58:00.183 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T17:58:00.183 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T17:58:00.184 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:58:00.184 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T17:58:00.184 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T17:58:00.185 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:58:00.185 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T17:58:00.186 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T17:58:00.186 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-20T17:58:00.187 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T17:58:00.187 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:00.187 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:58:00.187 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.32971 2026-02-20T17:58:00.188 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T17:58:00.188 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T17:58:00.188 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-20T17:58:00.189 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-20T17:58:00.189 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:00.189 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:58:00.189 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.32971 2026-02-20T17:58:00.190 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-20T17:58:00.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-20T17:58:00.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-20T17:58:00.192 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-20T17:58:00.192 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=shec_generic 2026-02-20T17:58:00.192 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-20T17:58:00.192 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-20T17:58:00.192 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-20T17:58:00.192 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-20T17:58:00.192 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-20T17:58:00.192 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=shec_generic 2026-02-20T17:58:00.210 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-20T17:58:00.211 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:58:00.211 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:58:00.211 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:58:00.211 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:00.211 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:58:00.211 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:58:00.212 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=shec_generic 2026-02-20T17:58:00.236 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-20T17:58:00.238 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-20T17:58:00.238 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T17:58:00.238 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T17:58:00.238 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-20T17:58:00.238 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-20T17:58:00.238 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T17:58:00.238 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:58:00.238 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:58:00.239 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:58:00.239 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:00.240 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:58:00.240 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:58:00.240 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T17:58:00.240 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.32971/ceph-mon.a.asok config get fsid 2026-02-20T17:58:00.282 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-20T17:58:00.282 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T17:58:00.282 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T17:58:00.282 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-20T17:58:00.283 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-20T17:58:00.284 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T17:58:00.284 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:58:00.284 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:58:00.284 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:58:00.284 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:00.284 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:58:00.284 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:58:00.284 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T17:58:00.284 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.32971/ceph-mon.a.asok config get mon_host 2026-02-20T17:58:00.328 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-20T17:58:00.328 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-20T17:58:00.328 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-20T17:58:00.328 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-20T17:58:00.328 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-20T17:58:00.328 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-20T17:58:00.329 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-20T17:58:00.425 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-20T17:58:00.425 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:58:00.426 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:58:00.426 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:58:00.426 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:00.426 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:58:00.426 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:58:00.426 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T17:58:00.427 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T17:58:00.444 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-20T17:58:00.445 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:58:00.445 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:58:00.445 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:58:00.445 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:00.445 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:58:00.445 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:58:00.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-20T17:58:00.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.32971/ceph-mon.a.asok log flush 2026-02-20T17:58:00.485 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T17:58:00.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=shec_generic 2026-02-20T17:58:00.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-20T17:58:00.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T17:58:00.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-20T17:58:00.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T17:58:00.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-20T17:58:00.493 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-20T17:58:00.493 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T17:58:00.493 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T17:58:00.493 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T17:58:00.493 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-20T17:58:00.493 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-20T17:58:00.493 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T17:58:00.493 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T17:58:00.493 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-20T17:58:00.493 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T17:58:00.493 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:58:00.493 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:58:00.493 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:58:00.494 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:00.494 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:58:00.494 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:58:00.494 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:58:00.494 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T17:58:00.494 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T17:58:00.494 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T17:58:00.494 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-20T17:58:00.494 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-20T17:58:00.494 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T17:58:00.494 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T17:58:00.494 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T17:58:00.495 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T17:58:00.495 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T17:58:00.495 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=shec_generic 2026-02-20T17:58:00.495 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-20T17:58:00.495 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T17:58:00.495 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f59db904-1583-411a-86bd-497189ec717b 2026-02-20T17:58:00.496 INFO:tasks.workunit.client.0.trial195.stdout:add osd0 f59db904-1583-411a-86bd-497189ec717b 2026-02-20T17:58:00.496 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 f59db904-1583-411a-86bd-497189ec717b' 2026-02-20T17:58:00.496 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T17:58:00.504 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCooJhpG2PyHRAAt299Bk73cbcCkbckARM9lA== 2026-02-20T17:58:00.504 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCooJhpG2PyHRAAt299Bk73cbcCkbckARM9lA=="}' 2026-02-20T17:58:00.504 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f59db904-1583-411a-86bd-497189ec717b -i td/test-erasure-code-plugins/0/new.json 2026-02-20T17:58:00.605 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T17:58:00.612 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-20T17:58:00.613 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_generic --mkfs --key AQCooJhpG2PyHRAAt299Bk73cbcCkbckARM9lA== --osd-uuid f59db904-1583-411a-86bd-497189ec717b 2026-02-20T17:58:00.631 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:00.630+0000 7f48d352d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:00.632 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:00.631+0000 7f48d352d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:00.634 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:00.632+0000 7f48d352d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:00.634 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:00.633+0000 7f48d352d900 -1 bdev(0x55ef01730c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-20T17:58:00.634 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:00.633+0000 7f48d352d900 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-20T17:58:01.059 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-20T17:58:01.059 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T17:58:01.060 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-20T17:58:01.060 INFO:tasks.workunit.client.0.trial195.stdout:adding osd0 key to auth repository 2026-02-20T17:58:01.061 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T17:58:01.164 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-20T17:58:01.165 INFO:tasks.workunit.client.0.trial195.stdout:start osd.0 2026-02-20T17:58:01.166 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_generic 2026-02-20T17:58:01.166 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T17:58:01.166 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T17:58:01.166 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T17:58:01.181 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:01.180+0000 7f68c2a13900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:01.182 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:01.181+0000 7f68c2a13900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:01.183 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:01.182+0000 7f68c2a13900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:01.263 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-20T17:58:01.263 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T17:58:01.263 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-20T17:58:01.264 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T17:58:01.264 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T17:58:01.264 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:58:01.264 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T17:58:01.265 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T17:58:01.265 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:58:01.265 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T17:58:01.363 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T17:58:01.399 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:01.398+0000 7f68c2a13900 -1 Falling back to public interface 2026-02-20T17:58:01.771 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:01.770+0000 7f68c2a13900 -1 osd.0 0 log_to_monitors true 2026-02-20T17:58:02.364 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T17:58:02.364 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:58:02.364 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T17:58:02.365 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T17:58:02.365 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:58:02.365 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T17:58:02.463 INFO:tasks.workunit.client.0.trial195.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1174763887,v1:127.0.0.1:6801/1174763887] [v2:127.0.0.1:6802/1174763887,v1:127.0.0.1:6803/1174763887] exists,up f59db904-1583-411a-86bd-497189ec717b 2026-02-20T17:58:02.464 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T17:58:02.464 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T17:58:02.464 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T17:58:02.465 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-20T17:58:02.465 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-20T17:58:02.465 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-20T17:58:02.465 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:58:02.465 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:02.465 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:58:02.465 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-osd.0.asok 2026-02-20T17:58:02.465 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-20T17:58:02.465 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.32971/ceph-osd.0.asok log flush 2026-02-20T17:58:02.512 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T17:58:02.516 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_generic' td/test-erasure-code-plugins/mon.a.log 2026-02-20T17:58:02.517 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T17:58:00.227+0000 7f38785c9d80 0 WARNING: osd_erasure_code_plugins contains plugin shec_generic that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-20T17:58:02.518 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_generic' td/test-erasure-code-plugins/osd.0.log 2026-02-20T17:58:02.519 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T17:58:01.398+0000 7f68c2a13900 0 WARNING: osd_erasure_code_plugins contains plugin shec_generic that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-20T17:58:02.521 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-20T17:58:02.521 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-20T17:58:02.522 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T17:58:02.522 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-20T17:58:02.522 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T17:58:02.522 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T17:58:02.522 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T17:58:02.522 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T17:58:02.522 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T17:58:02.625 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T17:58:02.625 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T17:58:02.626 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T17:58:02.627 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T17:58:02.627 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T17:58:02.627 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T17:58:02.628 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T17:58:02.628 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:58:02.628 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T17:58:02.629 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T17:58:02.629 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:58:02.629 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T17:58:02.631 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T17:58:02.631 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-20T17:58:02.634 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T17:58:02.634 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:02.634 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:58:02.634 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.32971 2026-02-20T17:58:02.636 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T17:58:02.637 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T17:58:02.637 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-20T17:58:02.637 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-20T17:58:02.638 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-20T17:58:02.638 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-20T17:58:02.638 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-20T17:58:02.638 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T17:58:02.638 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-20T17:58:02.638 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T17:58:02.638 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T17:58:02.638 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T17:58:02.638 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T17:58:02.638 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T17:58:02.638 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T17:58:02.638 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T17:58:02.638 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T17:58:02.638 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T17:58:02.638 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T17:58:02.638 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T17:58:02.639 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T17:58:02.639 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:58:02.639 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T17:58:02.640 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T17:58:02.640 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:58:02.641 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T17:58:02.642 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T17:58:02.642 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-20T17:58:02.643 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T17:58:02.643 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:02.643 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:58:02.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.32971 2026-02-20T17:58:02.644 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T17:58:02.644 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T17:58:02.644 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-20T17:58:02.645 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-20T17:58:02.645 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:02.645 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:58:02.645 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.32971 2026-02-20T17:58:02.646 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-20T17:58:02.647 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-20T17:58:02.647 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-20T17:58:02.647 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-20T17:58:02.647 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=shec_sse3 2026-02-20T17:58:02.647 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-20T17:58:02.647 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-20T17:58:02.647 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-20T17:58:02.648 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-20T17:58:02.648 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-20T17:58:02.648 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=shec_sse3 2026-02-20T17:58:02.668 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-20T17:58:02.668 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:58:02.668 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:58:02.668 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:58:02.668 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:02.668 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:58:02.668 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:58:02.668 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=shec_sse3 2026-02-20T17:58:02.693 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-20T17:58:02.694 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-20T17:58:02.694 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T17:58:02.694 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T17:58:02.694 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-20T17:58:02.695 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-20T17:58:02.695 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T17:58:02.695 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:58:02.695 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:58:02.696 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:58:02.696 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:02.696 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:58:02.696 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:58:02.696 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T17:58:02.696 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.32971/ceph-mon.a.asok config get fsid 2026-02-20T17:58:02.742 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-20T17:58:02.743 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T17:58:02.743 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T17:58:02.743 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-20T17:58:02.743 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-20T17:58:02.743 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T17:58:02.743 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:58:02.743 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:58:02.743 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:58:02.743 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:02.743 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:58:02.744 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:58:02.744 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T17:58:02.744 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.32971/ceph-mon.a.asok config get mon_host 2026-02-20T17:58:02.788 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-20T17:58:02.788 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-20T17:58:02.789 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-20T17:58:02.789 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-20T17:58:02.789 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-20T17:58:02.789 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-20T17:58:02.789 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-20T17:58:02.886 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-20T17:58:02.886 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:58:02.886 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:58:02.886 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:58:02.886 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:02.886 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:58:02.887 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:58:02.887 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T17:58:02.887 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T17:58:02.905 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-20T17:58:02.906 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:58:02.906 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:58:02.906 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:58:02.906 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:02.906 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:58:02.906 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:58:02.906 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-20T17:58:02.906 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.32971/ceph-mon.a.asok log flush 2026-02-20T17:58:02.945 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T17:58:02.952 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=shec_sse3 2026-02-20T17:58:02.952 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-20T17:58:02.952 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T17:58:02.952 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-20T17:58:02.952 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T17:58:02.952 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-20T17:58:02.952 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-20T17:58:02.952 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T17:58:02.953 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T17:58:02.953 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T17:58:02.953 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-20T17:58:02.953 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-20T17:58:02.953 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T17:58:02.953 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T17:58:02.953 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-20T17:58:02.953 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T17:58:02.953 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:58:02.953 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:58:02.953 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:58:02.953 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:02.953 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:58:02.954 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:58:02.954 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:58:02.954 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T17:58:02.954 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T17:58:02.954 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T17:58:02.954 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-20T17:58:02.954 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-20T17:58:02.954 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T17:58:02.954 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T17:58:02.954 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T17:58:02.954 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T17:58:02.954 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T17:58:02.955 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=shec_sse3 2026-02-20T17:58:02.955 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-20T17:58:02.955 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T17:58:02.955 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c7a0d6f9-589f-4466-a24f-3dbac6a57468 2026-02-20T17:58:02.955 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 c7a0d6f9-589f-4466-a24f-3dbac6a57468' 2026-02-20T17:58:02.956 INFO:tasks.workunit.client.0.trial195.stdout:add osd0 c7a0d6f9-589f-4466-a24f-3dbac6a57468 2026-02-20T17:58:02.956 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T17:58:02.964 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCqoJhp1NNnORAA5De7Xm9I6wEoStcYzXnR8Q== 2026-02-20T17:58:02.964 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCqoJhp1NNnORAA5De7Xm9I6wEoStcYzXnR8Q=="}' 2026-02-20T17:58:02.965 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c7a0d6f9-589f-4466-a24f-3dbac6a57468 -i td/test-erasure-code-plugins/0/new.json 2026-02-20T17:58:03.064 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T17:58:03.070 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-20T17:58:03.071 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_sse3 --mkfs --key AQCqoJhp1NNnORAA5De7Xm9I6wEoStcYzXnR8Q== --osd-uuid c7a0d6f9-589f-4466-a24f-3dbac6a57468 2026-02-20T17:58:03.089 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:03.088+0000 7f1584764900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:03.091 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:03.089+0000 7f1584764900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:03.092 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:03.090+0000 7f1584764900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:03.092 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:03.091+0000 7f1584764900 -1 bdev(0x55a3238c0c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-20T17:58:03.092 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:03.091+0000 7f1584764900 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-20T17:58:03.525 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-20T17:58:03.525 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T17:58:03.526 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-20T17:58:03.526 INFO:tasks.workunit.client.0.trial195.stdout:adding osd0 key to auth repository 2026-02-20T17:58:03.526 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T17:58:03.629 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-20T17:58:03.630 INFO:tasks.workunit.client.0.trial195.stdout:start osd.0 2026-02-20T17:58:03.631 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_sse3 2026-02-20T17:58:03.631 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T17:58:03.631 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T17:58:03.631 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T17:58:03.645 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:03.644+0000 7faaa8f44900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:03.646 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:03.645+0000 7faaa8f44900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:03.647 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:03.646+0000 7faaa8f44900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:03.729 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-20T17:58:03.729 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T17:58:03.729 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-20T17:58:03.729 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T17:58:03.729 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T17:58:03.729 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:58:03.729 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T17:58:03.730 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T17:58:03.730 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:58:03.730 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T17:58:03.826 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T17:58:03.849 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:03.848+0000 7faaa8f44900 -1 Falling back to public interface 2026-02-20T17:58:04.146 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:04.144+0000 7faaa8f44900 -1 osd.0 0 log_to_monitors true 2026-02-20T17:58:04.827 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T17:58:04.827 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:58:04.827 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T17:58:04.827 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T17:58:04.827 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:58:04.827 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T17:58:04.923 INFO:tasks.workunit.client.0.trial195.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2500808324,v1:127.0.0.1:6801/2500808324] [v2:127.0.0.1:6802/2500808324,v1:127.0.0.1:6803/2500808324] exists,up c7a0d6f9-589f-4466-a24f-3dbac6a57468 2026-02-20T17:58:04.924 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T17:58:04.925 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T17:58:04.925 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T17:58:04.925 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-20T17:58:04.925 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-20T17:58:04.925 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-20T17:58:04.925 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:58:04.925 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:04.925 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:58:04.925 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-osd.0.asok 2026-02-20T17:58:04.925 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-20T17:58:04.926 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.32971/ceph-osd.0.asok log flush 2026-02-20T17:58:04.971 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T17:58:04.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_sse3' td/test-erasure-code-plugins/mon.a.log 2026-02-20T17:58:04.976 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T17:58:02.683+0000 7f4adf3c8d80 0 WARNING: osd_erasure_code_plugins contains plugin shec_sse3 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-20T17:58:04.976 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_sse3' td/test-erasure-code-plugins/osd.0.log 2026-02-20T17:58:04.977 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T17:58:03.848+0000 7faaa8f44900 0 WARNING: osd_erasure_code_plugins contains plugin shec_sse3 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-20T17:58:04.978 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-20T17:58:04.978 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-20T17:58:04.979 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T17:58:04.979 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-20T17:58:04.979 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T17:58:04.979 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T17:58:04.979 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T17:58:04.979 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T17:58:04.979 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T17:58:05.085 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T17:58:05.086 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T17:58:05.086 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T17:58:05.087 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T17:58:05.088 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T17:58:05.088 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T17:58:05.088 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T17:58:05.089 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:58:05.090 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T17:58:05.090 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T17:58:05.090 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:58:05.091 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T17:58:05.092 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T17:58:05.092 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-20T17:58:05.095 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T17:58:05.095 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:05.095 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:58:05.095 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.32971 2026-02-20T17:58:05.097 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T17:58:05.097 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T17:58:05.097 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-20T17:58:05.097 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-20T17:58:05.097 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-20T17:58:05.097 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-20T17:58:05.097 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-20T17:58:05.098 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T17:58:05.098 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-20T17:58:05.098 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T17:58:05.098 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T17:58:05.098 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T17:58:05.098 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T17:58:05.098 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T17:58:05.099 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T17:58:05.099 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T17:58:05.099 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T17:58:05.099 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T17:58:05.100 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T17:58:05.100 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T17:58:05.101 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T17:58:05.101 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:58:05.101 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T17:58:05.102 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T17:58:05.102 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:58:05.102 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T17:58:05.103 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T17:58:05.104 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-20T17:58:05.105 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T17:58:05.105 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:05.105 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:58:05.105 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.32971 2026-02-20T17:58:05.106 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T17:58:05.106 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T17:58:05.106 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-20T17:58:05.107 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-20T17:58:05.107 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:05.107 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:58:05.107 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.32971 2026-02-20T17:58:05.108 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-20T17:58:05.109 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-20T17:58:05.109 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-20T17:58:05.109 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-20T17:58:05.109 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=shec_sse4 2026-02-20T17:58:05.109 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-20T17:58:05.110 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-20T17:58:05.110 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-20T17:58:05.110 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-20T17:58:05.110 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-20T17:58:05.110 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=shec_sse4 2026-02-20T17:58:05.129 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-20T17:58:05.129 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:58:05.129 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:58:05.129 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:58:05.130 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:05.130 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:58:05.130 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:58:05.130 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=shec_sse4 2026-02-20T17:58:05.154 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-20T17:58:05.155 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-20T17:58:05.155 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T17:58:05.155 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T17:58:05.155 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-20T17:58:05.155 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-20T17:58:05.156 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T17:58:05.156 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:58:05.156 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:58:05.157 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:58:05.157 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:05.157 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:58:05.157 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:58:05.157 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T17:58:05.157 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.32971/ceph-mon.a.asok config get fsid 2026-02-20T17:58:05.199 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-20T17:58:05.200 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T17:58:05.200 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T17:58:05.200 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-20T17:58:05.200 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-20T17:58:05.200 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T17:58:05.200 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:58:05.200 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:58:05.201 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:58:05.201 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:05.201 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:58:05.201 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:58:05.201 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T17:58:05.201 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.32971/ceph-mon.a.asok config get mon_host 2026-02-20T17:58:05.246 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-20T17:58:05.246 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-20T17:58:05.246 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-20T17:58:05.246 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-20T17:58:05.246 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-20T17:58:05.246 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-20T17:58:05.246 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-20T17:58:05.342 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-20T17:58:05.342 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:58:05.342 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:58:05.342 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:58:05.342 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:05.342 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:58:05.342 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:58:05.343 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T17:58:05.343 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T17:58:05.361 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-20T17:58:05.361 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:58:05.361 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:58:05.361 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:58:05.361 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:05.361 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:58:05.361 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-mon.a.asok 2026-02-20T17:58:05.361 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-20T17:58:05.362 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.32971/ceph-mon.a.asok log flush 2026-02-20T17:58:05.403 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T17:58:05.410 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=shec_sse4 2026-02-20T17:58:05.410 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-20T17:58:05.410 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T17:58:05.410 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-20T17:58:05.410 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T17:58:05.411 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-20T17:58:05.411 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-20T17:58:05.411 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T17:58:05.411 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T17:58:05.411 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T17:58:05.411 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-20T17:58:05.411 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-20T17:58:05.411 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T17:58:05.411 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T17:58:05.411 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-20T17:58:05.411 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T17:58:05.411 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:58:05.412 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:58:05.412 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:58:05.412 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:05.412 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:58:05.412 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:58:05.412 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' 2026-02-20T17:58:05.412 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T17:58:05.412 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T17:58:05.412 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T17:58:05.412 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-20T17:58:05.412 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-20T17:58:05.412 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T17:58:05.413 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T17:58:05.413 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T17:58:05.413 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T17:58:05.413 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T17:58:05.413 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=shec_sse4 2026-02-20T17:58:05.413 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-20T17:58:05.413 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T17:58:05.413 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=921cd4f9-095c-4a68-b705-0c29b1bc0cdd 2026-02-20T17:58:05.413 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 921cd4f9-095c-4a68-b705-0c29b1bc0cdd' 2026-02-20T17:58:05.414 INFO:tasks.workunit.client.0.trial195.stdout:add osd0 921cd4f9-095c-4a68-b705-0c29b1bc0cdd 2026-02-20T17:58:05.414 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T17:58:05.421 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCtoJhp0WkNGRAA68lqSfqvTau+DbCzYJL+Sg== 2026-02-20T17:58:05.422 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCtoJhp0WkNGRAA68lqSfqvTau+DbCzYJL+Sg=="}' 2026-02-20T17:58:05.422 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 921cd4f9-095c-4a68-b705-0c29b1bc0cdd -i td/test-erasure-code-plugins/0/new.json 2026-02-20T17:58:05.526 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T17:58:05.532 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-20T17:58:05.533 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_sse4 --mkfs --key AQCtoJhp0WkNGRAA68lqSfqvTau+DbCzYJL+Sg== --osd-uuid 921cd4f9-095c-4a68-b705-0c29b1bc0cdd 2026-02-20T17:58:05.551 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:05.549+0000 7fc8adf62900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:05.552 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:05.551+0000 7fc8adf62900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:05.553 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:05.552+0000 7fc8adf62900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:05.554 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:05.552+0000 7fc8adf62900 -1 bdev(0x55713dd14c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-20T17:58:05.554 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:05.552+0000 7fc8adf62900 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-20T17:58:05.958 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-20T17:58:05.958 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T17:58:05.958 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-20T17:58:05.959 INFO:tasks.workunit.client.0.trial195.stdout:adding osd0 key to auth repository 2026-02-20T17:58:05.959 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T17:58:06.060 INFO:tasks.workunit.client.0.trial195.stdout:start osd.0 2026-02-20T17:58:06.061 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-20T17:58:06.061 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=ef9b43ea-4322-410b-a244-3d9cee79ab55 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.32971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_sse4 2026-02-20T17:58:06.061 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T17:58:06.061 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T17:58:06.061 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T17:58:06.076 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:06.075+0000 7ffa91210900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:06.077 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:06.076+0000 7ffa91210900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:06.078 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:06.077+0000 7ffa91210900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:06.158 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-20T17:58:06.158 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T17:58:06.158 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-20T17:58:06.158 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T17:58:06.159 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T17:58:06.159 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:58:06.159 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T17:58:06.159 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T17:58:06.159 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:58:06.160 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T17:58:06.258 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T17:58:06.284 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:06.282+0000 7ffa91210900 -1 Falling back to public interface 2026-02-20T17:58:06.630 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:06.628+0000 7ffa91210900 -1 osd.0 0 log_to_monitors true 2026-02-20T17:58:07.259 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T17:58:07.260 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:58:07.260 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T17:58:07.261 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T17:58:07.261 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:58:07.261 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T17:58:07.358 INFO:tasks.workunit.client.0.trial195.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1469655586,v1:127.0.0.1:6801/1469655586] [v2:127.0.0.1:6802/1469655586,v1:127.0.0.1:6803/1469655586] exists,up 921cd4f9-095c-4a68-b705-0c29b1bc0cdd 2026-02-20T17:58:07.360 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T17:58:07.360 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T17:58:07.360 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T17:58:07.360 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-20T17:58:07.360 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-20T17:58:07.360 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-20T17:58:07.360 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:58:07.360 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:07.361 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:58:07.361 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.32971/ceph-osd.0.asok 2026-02-20T17:58:07.361 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-20T17:58:07.361 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.32971/ceph-osd.0.asok log flush 2026-02-20T17:58:07.407 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T17:58:07.411 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_sse4' td/test-erasure-code-plugins/mon.a.log 2026-02-20T17:58:07.412 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T17:58:05.144+0000 7f46c9428d80 0 WARNING: osd_erasure_code_plugins contains plugin shec_sse4 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-20T17:58:07.413 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_sse4' td/test-erasure-code-plugins/osd.0.log 2026-02-20T17:58:07.414 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T17:58:06.283+0000 7ffa91210900 0 WARNING: osd_erasure_code_plugins contains plugin shec_sse4 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-20T17:58:07.415 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-20T17:58:07.415 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-20T17:58:07.415 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T17:58:07.416 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-20T17:58:07.416 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T17:58:07.416 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T17:58:07.416 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T17:58:07.416 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T17:58:07.416 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T17:58:07.521 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T17:58:07.522 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T17:58:07.522 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T17:58:07.523 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T17:58:07.524 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T17:58:07.524 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T17:58:07.524 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T17:58:07.525 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:58:07.525 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T17:58:07.526 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T17:58:07.526 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:58:07.527 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T17:58:07.528 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T17:58:07.528 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-20T17:58:07.531 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T17:58:07.531 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:07.531 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:58:07.532 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.32971 2026-02-20T17:58:07.533 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T17:58:07.533 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T17:58:07.533 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:54: TEST_preload_warning: return 0 2026-02-20T17:58:07.533 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2401: main: code=0 2026-02-20T17:58:07.534 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: teardown td/test-erasure-code-plugins 0 2026-02-20T17:58:07.534 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-20T17:58:07.534 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-20T17:58:07.534 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-20T17:58:07.534 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T17:58:07.534 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T17:58:07.534 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T17:58:07.534 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T17:58:07.534 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T17:58:07.535 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T17:58:07.535 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T17:58:07.535 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T17:58:07.535 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T17:58:07.536 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T17:58:07.536 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T17:58:07.537 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T17:58:07.537 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:58:07.537 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T17:58:07.538 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T17:58:07.538 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:58:07.538 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T17:58:07.539 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-20T17:58:07.539 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-20T17:58:07.540 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T17:58:07.540 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:07.541 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.32971 2026-02-20T17:58:07.541 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.32971 2026-02-20T17:58:07.542 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T17:58:07.542 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T17:58:07.542 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2406: main: return 0 2026-02-20T17:58:07.542 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-20T17:58:07.542 DEBUG:teuthology.orchestra.run.trial195:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-20T17:58:07.567 INFO:tasks.workunit:Running workunit erasure-code/test-erasure-code.sh... 2026-02-20T17:58:07.567 DEBUG:teuthology.orchestra.run.trial195:workunit test erasure-code/test-erasure-code.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f7870454149f22574086dd2ff55d050585136531 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh 2026-02-20T17:58:07.629 INFO:tasks.workunit.client.0.trial195.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-20T17:58:07.634 INFO:tasks.workunit.client.0.trial195.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-20T17:58:07.634 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-20T17:58:07.635 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-20T17:58:07.635 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export PYTHONWARNINGS=ignore 2026-02-20T17:58:07.635 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: PYTHONWARNINGS=ignore 2026-02-20T17:58:07.635 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: export CEPH_CONF=/dev/null 2026-02-20T17:58:07.635 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: CEPH_CONF=/dev/null 2026-02-20T17:58:07.635 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2397: main: unset CEPH_ARGS 2026-02-20T17:58:07.635 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: local code 2026-02-20T17:58:07.635 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: run td/test-erasure-code 2026-02-20T17:58:07.635 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:22: run: local dir=td/test-erasure-code 2026-02-20T17:58:07.635 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:23: run: shift 2026-02-20T17:58:07.635 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:25: run: export CEPH_MON=127.0.0.1:7101 2026-02-20T17:58:07.636 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:25: run: CEPH_MON=127.0.0.1:7101 2026-02-20T17:58:07.636 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:26: run: export CEPH_ARGS 2026-02-20T17:58:07.636 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:27: run: uuidgen 2026-02-20T17:58:07.636 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:27: run: CEPH_ARGS+='--fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none ' 2026-02-20T17:58:07.636 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:28: run: CEPH_ARGS+='--mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-20T17:58:07.636 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:30: run: setup td/test-erasure-code 2026-02-20T17:58:07.636 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code 2026-02-20T17:58:07.636 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code 2026-02-20T17:58:07.636 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code 2026-02-20T17:58:07.636 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T17:58:07.636 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code KILL 2026-02-20T17:58:07.636 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T17:58:07.636 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T17:58:07.637 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T17:58:07.637 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T17:58:07.637 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T17:58:07.638 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T17:58:07.638 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T17:58:07.638 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T17:58:07.638 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T17:58:07.638 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T17:58:07.638 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T17:58:07.638 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T17:58:07.639 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:58:07.639 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T17:58:07.639 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T17:58:07.640 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T17:58:07.640 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T17:58:07.640 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T17:58:07.640 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code 2026-02-20T17:58:07.641 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T17:58:07.642 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:07.642 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.58348 2026-02-20T17:58:07.642 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.58348 2026-02-20T17:58:07.642 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T17:58:07.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T17:58:07.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code 2026-02-20T17:58:07.644 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-20T17:58:07.644 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:07.644 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.58348 2026-02-20T17:58:07.644 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.58348 2026-02-20T17:58:07.644 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-20T17:58:07.646 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-20T17:58:07.646 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-20T17:58:07.646 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-20T17:58:07.646 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code 1' TERM HUP INT 2026-02-20T17:58:07.646 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:31: run: run_mon td/test-erasure-code a 2026-02-20T17:58:07.646 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code 2026-02-20T17:58:07.646 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-20T17:58:07.646 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-20T17:58:07.646 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-20T17:58:07.646 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code/a 2026-02-20T17:58:07.647 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code/a --run-dir=td/test-erasure-code 2026-02-20T17:58:07.665 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-20T17:58:07.665 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:58:07.666 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:58:07.666 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:58:07.666 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:07.666 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.58348 2026-02-20T17:58:07.666 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.58348/$cluster-$name.asok' 2026-02-20T17:58:07.666 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code/a '--log-file=td/test-erasure-code/$name.log' '--admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code/log --run-dir=td/test-erasure-code '--pid-file=td/test-erasure-code/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-20T17:58:07.691 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-20T17:58:07.693 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-20T17:58:07.693 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T17:58:07.693 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T17:58:07.693 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-20T17:58:07.693 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-20T17:58:07.693 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T17:58:07.694 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:58:07.694 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:58:07.694 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:58:07.694 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:07.694 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.58348 2026-02-20T17:58:07.694 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.58348/ceph-mon.a.asok 2026-02-20T17:58:07.694 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T17:58:07.694 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.58348/ceph-mon.a.asok config get fsid 2026-02-20T17:58:07.739 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-20T17:58:07.739 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T17:58:07.739 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T17:58:07.739 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-20T17:58:07.739 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-20T17:58:07.739 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T17:58:07.740 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:58:07.740 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:58:07.740 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:58:07.740 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:07.740 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.58348 2026-02-20T17:58:07.740 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.58348/ceph-mon.a.asok 2026-02-20T17:58:07.740 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T17:58:07.740 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.58348/ceph-mon.a.asok config get mon_host 2026-02-20T17:58:07.782 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:32: run: run_mgr td/test-erasure-code x 2026-02-20T17:58:07.782 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code 2026-02-20T17:58:07.782 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-20T17:58:07.782 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-20T17:58:07.782 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-20T17:58:07.782 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code/x 2026-02-20T17:58:07.783 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-20T17:58:07.882 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-20T17:58:07.883 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:58:07.883 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:58:07.883 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:58:07.883 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:07.883 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.58348 2026-02-20T17:58:07.883 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.58348/$cluster-$name.asok' 2026-02-20T17:58:07.883 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T17:58:07.884 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code/x '--log-file=td/test-erasure-code/$name.log' '--admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' --run-dir=td/test-erasure-code '--pid-file=td/test-erasure-code/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T17:58:07.901 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:34: run: get_asok_path mon.a 2026-02-20T17:58:07.901 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T17:58:07.901 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T17:58:07.902 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:58:07.902 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:07.902 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.58348 2026-02-20T17:58:07.902 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.58348/ceph-mon.a.asok 2026-02-20T17:58:07.902 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:34: run: CEPH_ARGS= 2026-02-20T17:58:07.902 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:34: run: ceph --admin-daemon /tmp/ceph-asok.58348/ceph-mon.a.asok log flush 2026-02-20T17:58:07.942 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T17:58:07.946 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:35: run: grep 'load: jerasure.*lrc' td/test-erasure-code/mon.a.log 2026-02-20T17:58:07.947 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T17:58:07.683+0000 7f23d3a17d80 0 load: jerasure load: lrc 2026-02-20T17:58:07.948 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: seq 0 10 2026-02-20T17:58:07.952 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-20T17:58:07.953 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 0 2026-02-20T17:58:07.953 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-20T17:58:07.953 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T17:58:07.953 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-20T17:58:07.953 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T17:58:07.953 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/0 2026-02-20T17:58:07.953 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-20T17:58:07.953 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T17:58:07.953 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T17:58:07.954 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T17:58:07.954 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/0' 2026-02-20T17:58:07.954 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/0/journal' 2026-02-20T17:58:07.954 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T17:58:07.954 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T17:58:07.954 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-20T17:58:07.954 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T17:58:07.954 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:58:07.954 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:58:07.954 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:58:07.954 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:07.954 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.58348 2026-02-20T17:58:07.955 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.58348/$cluster-$name.asok' 2026-02-20T17:58:07.955 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' 2026-02-20T17:58:07.955 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T17:58:07.955 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T17:58:07.955 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T17:58:07.955 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-20T17:58:07.955 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-20T17:58:07.955 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T17:58:07.955 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T17:58:07.955 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T17:58:07.955 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T17:58:07.955 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T17:58:07.956 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T17:58:07.956 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/0 2026-02-20T17:58:07.956 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T17:58:07.956 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1114acc6-d9af-43bf-9f2d-b96f790926ce 2026-02-20T17:58:07.956 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 1114acc6-d9af-43bf-9f2d-b96f790926ce' 2026-02-20T17:58:07.956 INFO:tasks.workunit.client.0.trial195.stdout:add osd0 1114acc6-d9af-43bf-9f2d-b96f790926ce 2026-02-20T17:58:07.957 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T17:58:07.964 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCvoJhpKoxSORAArPa6B0o4J15wPamGMYlveg== 2026-02-20T17:58:07.964 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCvoJhpKoxSORAArPa6B0o4J15wPamGMYlveg=="}' 2026-02-20T17:58:07.964 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1114acc6-d9af-43bf-9f2d-b96f790926ce -i td/test-erasure-code/0/new.json 2026-02-20T17:58:08.065 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T17:58:08.072 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/0/new.json 2026-02-20T17:58:08.073 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/0 --osd-journal=td/test-erasure-code/0/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCvoJhpKoxSORAArPa6B0o4J15wPamGMYlveg== --osd-uuid 1114acc6-d9af-43bf-9f2d-b96f790926ce 2026-02-20T17:58:08.090 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:08.089+0000 7f37574e5900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:08.092 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:08.091+0000 7f37574e5900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:08.093 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:08.092+0000 7f37574e5900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:08.093 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:08.092+0000 7f37574e5900 -1 bdev(0x5559dfb12c00 td/test-erasure-code/0/block) open stat got: (1) Operation not permitted 2026-02-20T17:58:08.093 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:08.092+0000 7f37574e5900 -1 bluestore(td/test-erasure-code/0) _read_fsid unparsable uuid 2026-02-20T17:58:08.530 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/0/keyring 2026-02-20T17:58:08.530 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T17:58:08.531 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-20T17:58:08.531 INFO:tasks.workunit.client.0.trial195.stdout:adding osd0 key to auth repository 2026-02-20T17:58:08.531 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T17:58:08.636 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-20T17:58:08.636 INFO:tasks.workunit.client.0.trial195.stdout:start osd.0 2026-02-20T17:58:08.636 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/0 --osd-journal=td/test-erasure-code/0/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T17:58:08.636 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T17:58:08.636 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T17:58:08.636 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T17:58:08.652 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:08.652+0000 7f79bc070900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:08.654 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:08.653+0000 7f79bc070900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:08.655 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:08.654+0000 7f79bc070900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:08.736 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-20T17:58:08.736 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T17:58:08.736 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-20T17:58:08.736 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T17:58:08.737 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T17:58:08.737 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:58:08.737 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T17:58:08.737 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T17:58:08.738 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:58:08.738 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T17:58:08.833 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T17:58:08.867 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:08.865+0000 7f79bc070900 -1 Falling back to public interface 2026-02-20T17:58:09.325 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:09.324+0000 7f79bc070900 -1 osd.0 0 log_to_monitors true 2026-02-20T17:58:09.834 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T17:58:09.835 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:58:09.835 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T17:58:09.836 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T17:58:09.836 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:58:09.836 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T17:58:09.939 INFO:tasks.workunit.client.0.trial195.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2847111285,v1:127.0.0.1:6801/2847111285] [v2:127.0.0.1:6802/2847111285,v1:127.0.0.1:6803/2847111285] exists,up 1114acc6-d9af-43bf-9f2d-b96f790926ce 2026-02-20T17:58:09.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T17:58:09.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T17:58:09.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T17:58:09.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-20T17:58:09.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 1 2026-02-20T17:58:09.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-20T17:58:09.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T17:58:09.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-20T17:58:09.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T17:58:09.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/1 2026-02-20T17:58:09.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-20T17:58:09.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T17:58:09.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T17:58:09.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T17:58:09.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/1' 2026-02-20T17:58:09.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/1/journal' 2026-02-20T17:58:09.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T17:58:09.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T17:58:09.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-20T17:58:09.943 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T17:58:09.943 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:58:09.943 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:58:09.943 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:58:09.943 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:09.943 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.58348 2026-02-20T17:58:09.943 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.58348/$cluster-$name.asok' 2026-02-20T17:58:09.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' 2026-02-20T17:58:09.944 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T17:58:09.944 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T17:58:09.944 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T17:58:09.944 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-20T17:58:09.944 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-20T17:58:09.944 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T17:58:09.944 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T17:58:09.944 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T17:58:09.944 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T17:58:09.944 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T17:58:09.944 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T17:58:09.944 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/1 2026-02-20T17:58:09.944 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T17:58:09.944 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=7adcb234-8113-42a5-9160-ccc4daf31b11 2026-02-20T17:58:09.944 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 7adcb234-8113-42a5-9160-ccc4daf31b11' 2026-02-20T17:58:09.944 INFO:tasks.workunit.client.0.trial195.stdout:add osd1 7adcb234-8113-42a5-9160-ccc4daf31b11 2026-02-20T17:58:09.944 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T17:58:09.953 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCxoJhpPy3COBAAvQEY86QIoi9MhwfAGiuRBg== 2026-02-20T17:58:09.953 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCxoJhpPy3COBAAvQEY86QIoi9MhwfAGiuRBg=="}' 2026-02-20T17:58:09.953 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 7adcb234-8113-42a5-9160-ccc4daf31b11 -i td/test-erasure-code/1/new.json 2026-02-20T17:58:10.186 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T17:58:10.193 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/1/new.json 2026-02-20T17:58:10.194 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/1 --osd-journal=td/test-erasure-code/1/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCxoJhpPy3COBAAvQEY86QIoi9MhwfAGiuRBg== --osd-uuid 7adcb234-8113-42a5-9160-ccc4daf31b11 2026-02-20T17:58:10.213 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:10.212+0000 7feb81903900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:10.214 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:10.213+0000 7feb81903900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:10.215 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:10.214+0000 7feb81903900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:10.215 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:10.214+0000 7feb81903900 -1 bdev(0x556e0c29cc00 td/test-erasure-code/1/block) open stat got: (1) Operation not permitted 2026-02-20T17:58:10.215 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:10.215+0000 7feb81903900 -1 bluestore(td/test-erasure-code/1) _read_fsid unparsable uuid 2026-02-20T17:58:10.520 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/1/keyring 2026-02-20T17:58:10.520 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T17:58:10.521 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-20T17:58:10.521 INFO:tasks.workunit.client.0.trial195.stdout:adding osd1 key to auth repository 2026-02-20T17:58:10.521 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T17:58:10.834 INFO:tasks.workunit.client.0.trial195.stdout:start osd.1 2026-02-20T17:58:10.835 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-20T17:58:10.835 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/1 --osd-journal=td/test-erasure-code/1/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T17:58:10.835 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T17:58:10.835 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T17:58:10.835 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T17:58:10.852 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:10.851+0000 7f1d23219900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:10.853 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:10.852+0000 7f1d23219900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:10.854 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:10.853+0000 7f1d23219900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:10.987 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:10.985+0000 7f1d23219900 -1 Falling back to public interface 2026-02-20T17:58:11.060 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-20T17:58:11.060 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T17:58:11.060 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-20T17:58:11.060 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T17:58:11.060 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T17:58:11.061 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:58:11.061 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T17:58:11.061 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T17:58:11.061 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:58:11.062 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T17:58:11.281 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T17:58:11.294 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:11.292+0000 7f1d23219900 -1 osd.1 0 log_to_monitors true 2026-02-20T17:58:12.282 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T17:58:12.282 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:58:12.282 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T17:58:12.283 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T17:58:12.283 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:58:12.283 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T17:58:12.505 INFO:tasks.workunit.client.0.trial195.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1832887233,v1:127.0.0.1:6811/1832887233] [v2:127.0.0.1:6812/1832887233,v1:127.0.0.1:6813/1832887233] exists,up 7adcb234-8113-42a5-9160-ccc4daf31b11 2026-02-20T17:58:12.506 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T17:58:12.506 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T17:58:12.506 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T17:58:12.506 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-20T17:58:12.506 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 2 2026-02-20T17:58:12.506 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-20T17:58:12.506 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T17:58:12.506 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-20T17:58:12.506 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T17:58:12.506 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/2 2026-02-20T17:58:12.506 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-20T17:58:12.506 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T17:58:12.506 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T17:58:12.506 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T17:58:12.506 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/2' 2026-02-20T17:58:12.506 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/2/journal' 2026-02-20T17:58:12.506 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T17:58:12.506 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T17:58:12.506 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-20T17:58:12.507 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T17:58:12.507 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:58:12.507 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:58:12.507 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:58:12.507 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:12.507 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.58348 2026-02-20T17:58:12.507 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.58348/$cluster-$name.asok' 2026-02-20T17:58:12.507 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' 2026-02-20T17:58:12.507 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T17:58:12.507 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T17:58:12.507 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T17:58:12.507 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-20T17:58:12.507 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-20T17:58:12.508 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T17:58:12.508 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T17:58:12.508 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T17:58:12.508 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T17:58:12.508 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T17:58:12.508 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T17:58:12.508 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/2 2026-02-20T17:58:12.509 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T17:58:12.509 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=463955d2-aa42-4fa3-aed8-21715ded07c1 2026-02-20T17:58:12.509 INFO:tasks.workunit.client.0.trial195.stdout:add osd2 463955d2-aa42-4fa3-aed8-21715ded07c1 2026-02-20T17:58:12.509 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 463955d2-aa42-4fa3-aed8-21715ded07c1' 2026-02-20T17:58:12.510 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T17:58:12.523 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC0oJhpbHUVHxAAFsXet/MwEqiYe8NUoHJQ4Q== 2026-02-20T17:58:12.523 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC0oJhpbHUVHxAAFsXet/MwEqiYe8NUoHJQ4Q=="}' 2026-02-20T17:58:12.523 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 463955d2-aa42-4fa3-aed8-21715ded07c1 -i td/test-erasure-code/2/new.json 2026-02-20T17:58:12.752 INFO:tasks.workunit.client.0.trial195.stdout:2 2026-02-20T17:58:12.759 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/2/new.json 2026-02-20T17:58:12.761 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/2 --osd-journal=td/test-erasure-code/2/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC0oJhpbHUVHxAAFsXet/MwEqiYe8NUoHJQ4Q== --osd-uuid 463955d2-aa42-4fa3-aed8-21715ded07c1 2026-02-20T17:58:12.779 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:12.778+0000 7fe601a70900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:12.780 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:12.779+0000 7fe601a70900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:12.781 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:12.780+0000 7fe601a70900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:12.782 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:12.781+0000 7fe601a70900 -1 bdev(0x5560a4734c00 td/test-erasure-code/2/block) open stat got: (1) Operation not permitted 2026-02-20T17:58:12.782 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:12.781+0000 7fe601a70900 -1 bluestore(td/test-erasure-code/2) _read_fsid unparsable uuid 2026-02-20T17:58:13.101 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/2/keyring 2026-02-20T17:58:13.101 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T17:58:13.102 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-20T17:58:13.102 INFO:tasks.workunit.client.0.trial195.stdout:adding osd2 key to auth repository 2026-02-20T17:58:13.103 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T17:58:13.417 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-20T17:58:13.417 INFO:tasks.workunit.client.0.trial195.stdout:start osd.2 2026-02-20T17:58:13.418 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/2 --osd-journal=td/test-erasure-code/2/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T17:58:13.418 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T17:58:13.418 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T17:58:13.418 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T17:58:13.434 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:13.433+0000 7f19af305900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:13.435 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:13.434+0000 7f19af305900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:13.436 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:13.435+0000 7f19af305900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:13.558 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:13.557+0000 7f19af305900 -1 Falling back to public interface 2026-02-20T17:58:13.641 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-20T17:58:13.641 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T17:58:13.642 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-20T17:58:13.642 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T17:58:13.642 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T17:58:13.642 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:58:13.642 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T17:58:13.642 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T17:58:13.642 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:58:13.642 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T17:58:13.866 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T17:58:13.951 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:13.950+0000 7f19af305900 -1 osd.2 0 log_to_monitors true 2026-02-20T17:58:14.867 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T17:58:14.867 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:58:14.868 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T17:58:14.868 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T17:58:14.869 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:58:14.869 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T17:58:15.091 INFO:tasks.workunit.client.0.trial195.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3240177575,v1:127.0.0.1:6819/3240177575] [v2:127.0.0.1:6820/3240177575,v1:127.0.0.1:6821/3240177575] exists,up 463955d2-aa42-4fa3-aed8-21715ded07c1 2026-02-20T17:58:15.096 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T17:58:15.096 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T17:58:15.096 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T17:58:15.096 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-20T17:58:15.096 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 3 2026-02-20T17:58:15.096 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-20T17:58:15.096 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T17:58:15.096 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-20T17:58:15.096 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T17:58:15.097 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/3 2026-02-20T17:58:15.097 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-20T17:58:15.097 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T17:58:15.097 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T17:58:15.097 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T17:58:15.097 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/3' 2026-02-20T17:58:15.097 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/3/journal' 2026-02-20T17:58:15.097 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T17:58:15.097 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T17:58:15.097 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-20T17:58:15.097 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T17:58:15.097 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:58:15.098 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:58:15.098 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:58:15.098 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:15.098 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.58348 2026-02-20T17:58:15.098 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.58348/$cluster-$name.asok' 2026-02-20T17:58:15.098 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' 2026-02-20T17:58:15.098 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T17:58:15.098 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T17:58:15.098 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T17:58:15.098 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-20T17:58:15.098 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-20T17:58:15.098 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T17:58:15.099 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T17:58:15.099 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T17:58:15.099 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T17:58:15.099 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T17:58:15.099 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T17:58:15.099 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/3 2026-02-20T17:58:15.099 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T17:58:15.099 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=cecb896c-3ca9-406e-84e5-4e0aa187e970 2026-02-20T17:58:15.100 INFO:tasks.workunit.client.0.trial195.stdout:add osd3 cecb896c-3ca9-406e-84e5-4e0aa187e970 2026-02-20T17:58:15.100 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 cecb896c-3ca9-406e-84e5-4e0aa187e970' 2026-02-20T17:58:15.100 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T17:58:15.107 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC3oJhpAlRRBhAAQ6MbhuSQ5oI/0HIoEfameA== 2026-02-20T17:58:15.107 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC3oJhpAlRRBhAAQ6MbhuSQ5oI/0HIoEfameA=="}' 2026-02-20T17:58:15.107 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new cecb896c-3ca9-406e-84e5-4e0aa187e970 -i td/test-erasure-code/3/new.json 2026-02-20T17:58:15.336 INFO:tasks.workunit.client.0.trial195.stdout:3 2026-02-20T17:58:15.344 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/3/new.json 2026-02-20T17:58:15.345 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/3 --osd-journal=td/test-erasure-code/3/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC3oJhpAlRRBhAAQ6MbhuSQ5oI/0HIoEfameA== --osd-uuid cecb896c-3ca9-406e-84e5-4e0aa187e970 2026-02-20T17:58:15.363 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:15.361+0000 7f587f34f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:15.364 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:15.363+0000 7f587f34f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:15.365 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:15.364+0000 7f587f34f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:15.366 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:15.364+0000 7f587f34f900 -1 bdev(0x55f3bf5ecc00 td/test-erasure-code/3/block) open stat got: (1) Operation not permitted 2026-02-20T17:58:15.366 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:15.364+0000 7f587f34f900 -1 bluestore(td/test-erasure-code/3) _read_fsid unparsable uuid 2026-02-20T17:58:15.672 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/3/keyring 2026-02-20T17:58:15.672 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T17:58:15.673 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-20T17:58:15.673 INFO:tasks.workunit.client.0.trial195.stdout:adding osd3 key to auth repository 2026-02-20T17:58:15.674 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T17:58:15.992 INFO:tasks.workunit.client.0.trial195.stdout:start osd.3 2026-02-20T17:58:15.992 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-20T17:58:15.992 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/3 --osd-journal=td/test-erasure-code/3/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T17:58:15.992 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T17:58:15.992 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T17:58:15.992 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T17:58:16.008 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:16.007+0000 7fc71ac27900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:16.009 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:16.008+0000 7fc71ac27900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:16.010 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:16.009+0000 7fc71ac27900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:16.141 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:16.140+0000 7fc71ac27900 -1 Falling back to public interface 2026-02-20T17:58:16.213 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-20T17:58:16.213 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T17:58:16.213 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-20T17:58:16.213 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T17:58:16.213 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T17:58:16.214 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:58:16.214 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T17:58:16.214 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T17:58:16.214 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:58:16.215 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T17:58:16.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T17:58:16.478 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:16.477+0000 7fc71ac27900 -1 osd.3 0 log_to_monitors true 2026-02-20T17:58:17.439 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T17:58:17.439 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:58:17.439 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T17:58:17.439 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T17:58:17.439 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:58:17.439 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T17:58:17.664 INFO:tasks.workunit.client.0.trial195.stdout:osd.3 up in weight 1 up_from 19 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2393134929,v1:127.0.0.1:6827/2393134929] [v2:127.0.0.1:6828/2393134929,v1:127.0.0.1:6829/2393134929] exists,up cecb896c-3ca9-406e-84e5-4e0aa187e970 2026-02-20T17:58:17.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T17:58:17.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T17:58:17.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T17:58:17.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-20T17:58:17.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 4 2026-02-20T17:58:17.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-20T17:58:17.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T17:58:17.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-20T17:58:17.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T17:58:17.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/4 2026-02-20T17:58:17.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-20T17:58:17.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T17:58:17.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T17:58:17.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T17:58:17.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/4' 2026-02-20T17:58:17.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/4/journal' 2026-02-20T17:58:17.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T17:58:17.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T17:58:17.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-20T17:58:17.669 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T17:58:17.669 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:58:17.670 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:58:17.670 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:58:17.670 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:17.670 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.58348 2026-02-20T17:58:17.670 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.58348/$cluster-$name.asok' 2026-02-20T17:58:17.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' 2026-02-20T17:58:17.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T17:58:17.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T17:58:17.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T17:58:17.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-20T17:58:17.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-20T17:58:17.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T17:58:17.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T17:58:17.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T17:58:17.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T17:58:17.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T17:58:17.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T17:58:17.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/4 2026-02-20T17:58:17.670 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T17:58:17.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=371cf062-cfe2-4dc2-a787-8aff6172a4b9 2026-02-20T17:58:17.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 371cf062-cfe2-4dc2-a787-8aff6172a4b9' 2026-02-20T17:58:17.670 INFO:tasks.workunit.client.0.trial195.stdout:add osd4 371cf062-cfe2-4dc2-a787-8aff6172a4b9 2026-02-20T17:58:17.671 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T17:58:17.680 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC5oJhpAXV1KBAAedeKje0QJP+VnZ1l/ZrlkA== 2026-02-20T17:58:17.680 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC5oJhpAXV1KBAAedeKje0QJP+VnZ1l/ZrlkA=="}' 2026-02-20T17:58:17.681 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 371cf062-cfe2-4dc2-a787-8aff6172a4b9 -i td/test-erasure-code/4/new.json 2026-02-20T17:58:17.907 INFO:tasks.workunit.client.0.trial195.stdout:4 2026-02-20T17:58:17.914 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/4/new.json 2026-02-20T17:58:17.915 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/4 --osd-journal=td/test-erasure-code/4/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC5oJhpAXV1KBAAedeKje0QJP+VnZ1l/ZrlkA== --osd-uuid 371cf062-cfe2-4dc2-a787-8aff6172a4b9 2026-02-20T17:58:17.933 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:17.931+0000 7f3b6f0a1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:17.934 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:17.933+0000 7f3b6f0a1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:17.935 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:17.934+0000 7f3b6f0a1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:17.935 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:17.934+0000 7f3b6f0a1900 -1 bdev(0x562dc84e0c00 td/test-erasure-code/4/block) open stat got: (1) Operation not permitted 2026-02-20T17:58:17.936 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:17.934+0000 7f3b6f0a1900 -1 bluestore(td/test-erasure-code/4) _read_fsid unparsable uuid 2026-02-20T17:58:18.267 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/4/keyring 2026-02-20T17:58:18.267 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T17:58:18.268 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-20T17:58:18.268 INFO:tasks.workunit.client.0.trial195.stdout:adding osd4 key to auth repository 2026-02-20T17:58:18.268 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T17:58:18.580 INFO:tasks.workunit.client.0.trial195.stdout:start osd.4 2026-02-20T17:58:18.581 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-20T17:58:18.581 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/4 --osd-journal=td/test-erasure-code/4/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T17:58:18.582 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T17:58:18.582 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T17:58:18.582 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T17:58:18.597 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:18.596+0000 7ff57ded0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:18.598 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:18.597+0000 7ff57ded0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:18.599 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:18.598+0000 7ff57ded0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:18.715 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:18.714+0000 7ff57ded0900 -1 Falling back to public interface 2026-02-20T17:58:18.804 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-20T17:58:18.804 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T17:58:18.804 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-20T17:58:18.804 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T17:58:18.804 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T17:58:18.804 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:58:18.804 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T17:58:18.805 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T17:58:18.805 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:58:18.805 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-20T17:58:19.024 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T17:58:19.089 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:19.088+0000 7ff57ded0900 -1 osd.4 0 log_to_monitors true 2026-02-20T17:58:20.025 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T17:58:20.025 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:58:20.025 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T17:58:20.026 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T17:58:20.026 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:58:20.026 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-20T17:58:20.252 INFO:tasks.workunit.client.0.trial195.stdout:osd.4 up in weight 1 up_from 24 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/1271040522,v1:127.0.0.1:6835/1271040522] [v2:127.0.0.1:6836/1271040522,v1:127.0.0.1:6837/1271040522] exists,up 371cf062-cfe2-4dc2-a787-8aff6172a4b9 2026-02-20T17:58:20.257 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T17:58:20.257 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T17:58:20.257 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T17:58:20.257 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-20T17:58:20.257 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 5 2026-02-20T17:58:20.257 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-20T17:58:20.257 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T17:58:20.257 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-20T17:58:20.257 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T17:58:20.257 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/5 2026-02-20T17:58:20.258 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-20T17:58:20.258 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T17:58:20.258 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T17:58:20.258 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T17:58:20.258 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/5' 2026-02-20T17:58:20.258 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/5/journal' 2026-02-20T17:58:20.258 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T17:58:20.258 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T17:58:20.258 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-20T17:58:20.258 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T17:58:20.258 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:58:20.258 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:58:20.259 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:58:20.259 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:20.259 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.58348 2026-02-20T17:58:20.259 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.58348/$cluster-$name.asok' 2026-02-20T17:58:20.259 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' 2026-02-20T17:58:20.259 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T17:58:20.259 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T17:58:20.259 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T17:58:20.259 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-20T17:58:20.259 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-20T17:58:20.259 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T17:58:20.259 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T17:58:20.260 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T17:58:20.260 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T17:58:20.260 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T17:58:20.260 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T17:58:20.260 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/5 2026-02-20T17:58:20.260 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T17:58:20.260 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8d3a4486-c70e-46e7-976f-259ea195d131 2026-02-20T17:58:20.260 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 8d3a4486-c70e-46e7-976f-259ea195d131' 2026-02-20T17:58:20.260 INFO:tasks.workunit.client.0.trial195.stdout:add osd5 8d3a4486-c70e-46e7-976f-259ea195d131 2026-02-20T17:58:20.261 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T17:58:20.268 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC8oJhpI73tDxAAZGTHzsQBgXW2bA0yjKqbog== 2026-02-20T17:58:20.268 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC8oJhpI73tDxAAZGTHzsQBgXW2bA0yjKqbog=="}' 2026-02-20T17:58:20.269 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8d3a4486-c70e-46e7-976f-259ea195d131 -i td/test-erasure-code/5/new.json 2026-02-20T17:58:20.497 INFO:tasks.workunit.client.0.trial195.stdout:5 2026-02-20T17:58:20.505 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/5/new.json 2026-02-20T17:58:20.506 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/5 --osd-journal=td/test-erasure-code/5/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC8oJhpI73tDxAAZGTHzsQBgXW2bA0yjKqbog== --osd-uuid 8d3a4486-c70e-46e7-976f-259ea195d131 2026-02-20T17:58:20.523 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:20.522+0000 7fefd6853900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:20.525 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:20.524+0000 7fefd6853900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:20.525 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:20.524+0000 7fefd6853900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:20.526 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:20.525+0000 7fefd6853900 -1 bdev(0x5566710e2c00 td/test-erasure-code/5/block) open stat got: (1) Operation not permitted 2026-02-20T17:58:20.526 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:20.525+0000 7fefd6853900 -1 bluestore(td/test-erasure-code/5) _read_fsid unparsable uuid 2026-02-20T17:58:20.855 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/5/keyring 2026-02-20T17:58:20.855 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T17:58:20.856 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-20T17:58:20.856 INFO:tasks.workunit.client.0.trial195.stdout:adding osd5 key to auth repository 2026-02-20T17:58:20.856 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T17:58:21.172 INFO:tasks.workunit.client.0.trial195.stdout:start osd.5 2026-02-20T17:58:21.173 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-20T17:58:21.173 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/5 --osd-journal=td/test-erasure-code/5/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T17:58:21.173 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T17:58:21.173 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T17:58:21.173 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T17:58:21.189 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:21.188+0000 7f8e88c8f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:21.190 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:21.189+0000 7f8e88c8f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:21.191 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:21.190+0000 7f8e88c8f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:21.322 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:21.320+0000 7f8e88c8f900 -1 Falling back to public interface 2026-02-20T17:58:21.398 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-20T17:58:21.398 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T17:58:21.398 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-20T17:58:21.398 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T17:58:21.398 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T17:58:21.398 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:58:21.399 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T17:58:21.399 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T17:58:21.399 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:58:21.400 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-20T17:58:21.628 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T17:58:21.710 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:21.709+0000 7f8e88c8f900 -1 osd.5 0 log_to_monitors true 2026-02-20T17:58:22.630 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T17:58:22.630 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:58:22.630 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T17:58:22.631 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T17:58:22.631 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:58:22.631 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-20T17:58:22.854 INFO:tasks.workunit.client.0.trial195.stdout:osd.5 up in weight 1 up_from 28 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/4252316213,v1:127.0.0.1:6843/4252316213] [v2:127.0.0.1:6844/4252316213,v1:127.0.0.1:6845/4252316213] exists,up 8d3a4486-c70e-46e7-976f-259ea195d131 2026-02-20T17:58:22.858 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T17:58:22.858 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T17:58:22.859 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T17:58:22.859 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-20T17:58:22.859 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 6 2026-02-20T17:58:22.859 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-20T17:58:22.859 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T17:58:22.859 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-20T17:58:22.859 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T17:58:22.859 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/6 2026-02-20T17:58:22.859 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-20T17:58:22.859 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T17:58:22.859 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T17:58:22.860 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T17:58:22.860 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/6' 2026-02-20T17:58:22.860 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/6/journal' 2026-02-20T17:58:22.860 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T17:58:22.860 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T17:58:22.860 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-20T17:58:22.860 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T17:58:22.860 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:58:22.860 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:58:22.860 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:58:22.860 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:22.860 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.58348 2026-02-20T17:58:22.860 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.58348/$cluster-$name.asok' 2026-02-20T17:58:22.861 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' 2026-02-20T17:58:22.861 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T17:58:22.861 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T17:58:22.861 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T17:58:22.861 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-20T17:58:22.861 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-20T17:58:22.861 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T17:58:22.861 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T17:58:22.862 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T17:58:22.862 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T17:58:22.862 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T17:58:22.862 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T17:58:22.862 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/6 2026-02-20T17:58:22.862 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T17:58:22.862 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=fed68fe0-32a7-4c8f-b0e1-51e9d8edb65f 2026-02-20T17:58:22.862 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 fed68fe0-32a7-4c8f-b0e1-51e9d8edb65f' 2026-02-20T17:58:22.862 INFO:tasks.workunit.client.0.trial195.stdout:add osd6 fed68fe0-32a7-4c8f-b0e1-51e9d8edb65f 2026-02-20T17:58:22.863 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T17:58:22.870 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC+oJhpdLzFMxAAvSYZfbtuwe2j/DHS5KeTMw== 2026-02-20T17:58:22.870 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC+oJhpdLzFMxAAvSYZfbtuwe2j/DHS5KeTMw=="}' 2026-02-20T17:58:22.870 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new fed68fe0-32a7-4c8f-b0e1-51e9d8edb65f -i td/test-erasure-code/6/new.json 2026-02-20T17:58:23.098 INFO:tasks.workunit.client.0.trial195.stdout:6 2026-02-20T17:58:23.106 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/6/new.json 2026-02-20T17:58:23.107 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/6 --osd-journal=td/test-erasure-code/6/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC+oJhpdLzFMxAAvSYZfbtuwe2j/DHS5KeTMw== --osd-uuid fed68fe0-32a7-4c8f-b0e1-51e9d8edb65f 2026-02-20T17:58:23.125 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:23.124+0000 7f3bf7470900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:23.127 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:23.126+0000 7f3bf7470900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:23.127 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:23.126+0000 7f3bf7470900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:23.128 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:23.127+0000 7f3bf7470900 -1 bdev(0x56545b5d0c00 td/test-erasure-code/6/block) open stat got: (1) Operation not permitted 2026-02-20T17:58:23.128 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:23.127+0000 7f3bf7470900 -1 bluestore(td/test-erasure-code/6) _read_fsid unparsable uuid 2026-02-20T17:58:23.455 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/6/keyring 2026-02-20T17:58:23.455 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T17:58:23.455 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-20T17:58:23.456 INFO:tasks.workunit.client.0.trial195.stdout:adding osd6 key to auth repository 2026-02-20T17:58:23.456 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T17:58:23.766 INFO:tasks.workunit.client.0.trial195.stdout:start osd.6 2026-02-20T17:58:23.767 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-20T17:58:23.768 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/6 --osd-journal=td/test-erasure-code/6/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T17:58:23.768 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T17:58:23.768 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T17:58:23.768 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T17:58:23.783 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:23.782+0000 7f8d6d067900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:23.784 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:23.783+0000 7f8d6d067900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:23.785 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:23.784+0000 7f8d6d067900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:23.975 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:23.973+0000 7f8d6d067900 -1 Falling back to public interface 2026-02-20T17:58:23.994 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-20T17:58:23.994 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T17:58:23.995 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-20T17:58:23.995 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T17:58:23.995 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T17:58:23.995 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:58:23.995 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T17:58:23.995 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T17:58:23.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:58:23.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-20T17:58:24.217 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T17:58:24.292 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:24.290+0000 7f8d6d067900 -1 osd.6 0 log_to_monitors true 2026-02-20T17:58:25.219 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T17:58:25.219 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:58:25.219 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T17:58:25.220 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T17:58:25.220 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:58:25.220 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-20T17:58:25.451 INFO:tasks.workunit.client.0.trial195.stdout:osd.6 up in weight 1 up_from 33 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/3279443325,v1:127.0.0.1:6851/3279443325] [v2:127.0.0.1:6852/3279443325,v1:127.0.0.1:6853/3279443325] exists,up fed68fe0-32a7-4c8f-b0e1-51e9d8edb65f 2026-02-20T17:58:25.454 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T17:58:25.455 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T17:58:25.455 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T17:58:25.455 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-20T17:58:25.455 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 7 2026-02-20T17:58:25.455 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-20T17:58:25.455 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T17:58:25.455 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=7 2026-02-20T17:58:25.455 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T17:58:25.455 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/7 2026-02-20T17:58:25.455 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-20T17:58:25.456 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T17:58:25.456 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T17:58:25.456 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T17:58:25.456 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/7' 2026-02-20T17:58:25.456 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/7/journal' 2026-02-20T17:58:25.456 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T17:58:25.456 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T17:58:25.456 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-20T17:58:25.456 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T17:58:25.456 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:58:25.456 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:58:25.456 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:58:25.457 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:25.457 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.58348 2026-02-20T17:58:25.457 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.58348/$cluster-$name.asok' 2026-02-20T17:58:25.457 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' 2026-02-20T17:58:25.457 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T17:58:25.457 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T17:58:25.457 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T17:58:25.457 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-20T17:58:25.458 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-20T17:58:25.458 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T17:58:25.458 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T17:58:25.458 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T17:58:25.458 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T17:58:25.458 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T17:58:25.458 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T17:58:25.458 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/7 2026-02-20T17:58:25.458 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T17:58:25.458 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=edea5b45-ad15-4fe3-a51d-9320b2cd8a0e 2026-02-20T17:58:25.458 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd7 edea5b45-ad15-4fe3-a51d-9320b2cd8a0e' 2026-02-20T17:58:25.459 INFO:tasks.workunit.client.0.trial195.stdout:add osd7 edea5b45-ad15-4fe3-a51d-9320b2cd8a0e 2026-02-20T17:58:25.459 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T17:58:25.466 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDBoJhp38G6GxAAt1MlGwxbKD16HtdkQ78hfg== 2026-02-20T17:58:25.466 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDBoJhp38G6GxAAt1MlGwxbKD16HtdkQ78hfg=="}' 2026-02-20T17:58:25.466 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new edea5b45-ad15-4fe3-a51d-9320b2cd8a0e -i td/test-erasure-code/7/new.json 2026-02-20T17:58:25.697 INFO:tasks.workunit.client.0.trial195.stdout:7 2026-02-20T17:58:25.705 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/7/new.json 2026-02-20T17:58:25.706 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 7 --fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/7 --osd-journal=td/test-erasure-code/7/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDBoJhp38G6GxAAt1MlGwxbKD16HtdkQ78hfg== --osd-uuid edea5b45-ad15-4fe3-a51d-9320b2cd8a0e 2026-02-20T17:58:25.724 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:25.722+0000 7faa40918900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:25.725 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:25.723+0000 7faa40918900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:25.726 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:25.724+0000 7faa40918900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:25.727 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:25.725+0000 7faa40918900 -1 bdev(0x55c6ac5c8c00 td/test-erasure-code/7/block) open stat got: (1) Operation not permitted 2026-02-20T17:58:25.727 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:25.725+0000 7faa40918900 -1 bluestore(td/test-erasure-code/7) _read_fsid unparsable uuid 2026-02-20T17:58:26.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/7/keyring 2026-02-20T17:58:26.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T17:58:26.082 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd7 key to auth repository 2026-02-20T17:58:26.082 INFO:tasks.workunit.client.0.trial195.stdout:adding osd7 key to auth repository 2026-02-20T17:58:26.082 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/7/keyring auth add osd.7 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T17:58:26.396 INFO:tasks.workunit.client.0.trial195.stdout:start osd.7 2026-02-20T17:58:26.397 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.7 2026-02-20T17:58:26.397 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 7 --fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/7 --osd-journal=td/test-erasure-code/7/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T17:58:26.397 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T17:58:26.397 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T17:58:26.397 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T17:58:26.412 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:26.410+0000 7f3c36e0d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:26.413 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:26.411+0000 7f3c36e0d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:26.414 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:26.412+0000 7f3c36e0d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:26.550 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:26.547+0000 7f3c36e0d900 -1 Falling back to public interface 2026-02-20T17:58:26.621 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 7 2026-02-20T17:58:26.621 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T17:58:26.621 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=7 2026-02-20T17:58:26.621 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T17:58:26.621 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T17:58:26.621 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:58:26.622 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T17:58:26.622 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T17:58:26.623 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:58:26.623 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-20T17:58:26.844 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T17:58:26.917 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:26.914+0000 7f3c36e0d900 -1 osd.7 0 log_to_monitors true 2026-02-20T17:58:27.845 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T17:58:27.845 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:58:27.845 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T17:58:27.845 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T17:58:27.846 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:58:27.846 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-20T17:58:28.074 INFO:tasks.workunit.client.0.trial195.stdout:osd.7 up in weight 1 up_from 38 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6858/2748013885,v1:127.0.0.1:6859/2748013885] [v2:127.0.0.1:6860/2748013885,v1:127.0.0.1:6861/2748013885] exists,up edea5b45-ad15-4fe3-a51d-9320b2cd8a0e 2026-02-20T17:58:28.078 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T17:58:28.078 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T17:58:28.078 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T17:58:28.078 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-20T17:58:28.078 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 8 2026-02-20T17:58:28.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-20T17:58:28.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T17:58:28.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=8 2026-02-20T17:58:28.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T17:58:28.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/8 2026-02-20T17:58:28.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-20T17:58:28.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T17:58:28.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T17:58:28.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T17:58:28.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/8' 2026-02-20T17:58:28.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/8/journal' 2026-02-20T17:58:28.080 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T17:58:28.080 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T17:58:28.080 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-20T17:58:28.080 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T17:58:28.080 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:58:28.080 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:58:28.080 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:58:28.080 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:28.080 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.58348 2026-02-20T17:58:28.080 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.58348/$cluster-$name.asok' 2026-02-20T17:58:28.080 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' 2026-02-20T17:58:28.080 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T17:58:28.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T17:58:28.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T17:58:28.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-20T17:58:28.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-20T17:58:28.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T17:58:28.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T17:58:28.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T17:58:28.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T17:58:28.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T17:58:28.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T17:58:28.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/8 2026-02-20T17:58:28.081 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T17:58:28.082 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=14db9dbd-229c-4d1d-9e8b-076b70123c3e 2026-02-20T17:58:28.082 INFO:tasks.workunit.client.0.trial195.stdout:add osd8 14db9dbd-229c-4d1d-9e8b-076b70123c3e 2026-02-20T17:58:28.082 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd8 14db9dbd-229c-4d1d-9e8b-076b70123c3e' 2026-02-20T17:58:28.082 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T17:58:28.090 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDEoJhpYoBLBRAAmwm+YV9xxo7UkmauMlizTQ== 2026-02-20T17:58:28.090 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDEoJhpYoBLBRAAmwm+YV9xxo7UkmauMlizTQ=="}' 2026-02-20T17:58:28.090 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 14db9dbd-229c-4d1d-9e8b-076b70123c3e -i td/test-erasure-code/8/new.json 2026-02-20T17:58:28.317 INFO:tasks.workunit.client.0.trial195.stdout:8 2026-02-20T17:58:28.325 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/8/new.json 2026-02-20T17:58:28.326 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 8 --fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/8 --osd-journal=td/test-erasure-code/8/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDEoJhpYoBLBRAAmwm+YV9xxo7UkmauMlizTQ== --osd-uuid 14db9dbd-229c-4d1d-9e8b-076b70123c3e 2026-02-20T17:58:28.343 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:28.341+0000 7f2c5d905900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:28.344 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:28.342+0000 7f2c5d905900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:28.345 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:28.343+0000 7f2c5d905900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:28.346 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:28.344+0000 7f2c5d905900 -1 bdev(0x5560cfb2ec00 td/test-erasure-code/8/block) open stat got: (1) Operation not permitted 2026-02-20T17:58:28.346 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:28.344+0000 7f2c5d905900 -1 bluestore(td/test-erasure-code/8) _read_fsid unparsable uuid 2026-02-20T17:58:28.661 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/8/keyring 2026-02-20T17:58:28.661 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T17:58:28.661 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd8 key to auth repository 2026-02-20T17:58:28.662 INFO:tasks.workunit.client.0.trial195.stdout:adding osd8 key to auth repository 2026-02-20T17:58:28.662 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/8/keyring auth add osd.8 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T17:58:28.973 INFO:tasks.workunit.client.0.trial195.stdout:start osd.8 2026-02-20T17:58:28.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.8 2026-02-20T17:58:28.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 8 --fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/8 --osd-journal=td/test-erasure-code/8/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T17:58:28.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T17:58:28.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T17:58:28.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T17:58:28.989 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:28.987+0000 7f76df729900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:28.991 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:28.989+0000 7f76df729900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:28.992 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:28.990+0000 7f76df729900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:29.141 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:29.139+0000 7f76df729900 -1 Falling back to public interface 2026-02-20T17:58:29.195 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 8 2026-02-20T17:58:29.196 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T17:58:29.196 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=8 2026-02-20T17:58:29.196 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T17:58:29.196 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T17:58:29.196 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:58:29.196 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T17:58:29.196 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T17:58:29.197 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:58:29.197 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-20T17:58:29.420 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T17:58:29.462 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:29.460+0000 7f76df729900 -1 osd.8 0 log_to_monitors true 2026-02-20T17:58:30.421 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T17:58:30.421 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:58:30.421 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T17:58:30.422 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T17:58:30.422 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:58:30.422 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-20T17:58:30.647 INFO:tasks.workunit.client.0.trial195.stdout:osd.8 up in weight 1 up_from 42 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6866/2508145308,v1:127.0.0.1:6867/2508145308] [v2:127.0.0.1:6868/2508145308,v1:127.0.0.1:6869/2508145308] exists,up 14db9dbd-229c-4d1d-9e8b-076b70123c3e 2026-02-20T17:58:30.651 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T17:58:30.651 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T17:58:30.651 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T17:58:30.651 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-20T17:58:30.652 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 9 2026-02-20T17:58:30.652 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-20T17:58:30.652 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T17:58:30.652 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=9 2026-02-20T17:58:30.652 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T17:58:30.652 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/9 2026-02-20T17:58:30.652 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-20T17:58:30.652 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T17:58:30.652 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T17:58:30.652 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T17:58:30.652 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/9' 2026-02-20T17:58:30.653 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/9/journal' 2026-02-20T17:58:30.653 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T17:58:30.653 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T17:58:30.653 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-20T17:58:30.653 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T17:58:30.653 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:58:30.653 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:58:30.653 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:58:30.653 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:30.653 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.58348 2026-02-20T17:58:30.653 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.58348/$cluster-$name.asok' 2026-02-20T17:58:30.653 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' 2026-02-20T17:58:30.653 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T17:58:30.654 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T17:58:30.654 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T17:58:30.654 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-20T17:58:30.654 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-20T17:58:30.654 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T17:58:30.654 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T17:58:30.654 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T17:58:30.654 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T17:58:30.654 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T17:58:30.654 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T17:58:30.654 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/9 2026-02-20T17:58:30.654 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T17:58:30.655 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c0629070-187c-4594-aae4-4fd19faed37e 2026-02-20T17:58:30.655 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd9 c0629070-187c-4594-aae4-4fd19faed37e' 2026-02-20T17:58:30.655 INFO:tasks.workunit.client.0.trial195.stdout:add osd9 c0629070-187c-4594-aae4-4fd19faed37e 2026-02-20T17:58:30.655 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T17:58:30.663 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDGoJhpfi92JxAANWvp+/thm5M8hXM1QmX7Aw== 2026-02-20T17:58:30.663 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDGoJhpfi92JxAANWvp+/thm5M8hXM1QmX7Aw=="}' 2026-02-20T17:58:30.663 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c0629070-187c-4594-aae4-4fd19faed37e -i td/test-erasure-code/9/new.json 2026-02-20T17:58:30.889 INFO:tasks.workunit.client.0.trial195.stdout:9 2026-02-20T17:58:30.896 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/9/new.json 2026-02-20T17:58:30.897 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 9 --fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/9 --osd-journal=td/test-erasure-code/9/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDGoJhpfi92JxAANWvp+/thm5M8hXM1QmX7Aw== --osd-uuid c0629070-187c-4594-aae4-4fd19faed37e 2026-02-20T17:58:30.915 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:30.913+0000 7f436b2ec900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:30.916 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:30.914+0000 7f436b2ec900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:30.917 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:30.915+0000 7f436b2ec900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:30.918 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:30.915+0000 7f436b2ec900 -1 bdev(0x55d82d53ec00 td/test-erasure-code/9/block) open stat got: (1) Operation not permitted 2026-02-20T17:58:30.918 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:30.916+0000 7f436b2ec900 -1 bluestore(td/test-erasure-code/9) _read_fsid unparsable uuid 2026-02-20T17:58:31.244 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/9/keyring 2026-02-20T17:58:31.244 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T17:58:31.244 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd9 key to auth repository 2026-02-20T17:58:31.245 INFO:tasks.workunit.client.0.trial195.stdout:adding osd9 key to auth repository 2026-02-20T17:58:31.245 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/9/keyring auth add osd.9 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T17:58:31.554 INFO:tasks.workunit.client.0.trial195.stdout:start osd.9 2026-02-20T17:58:31.555 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.9 2026-02-20T17:58:31.555 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 9 --fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/9 --osd-journal=td/test-erasure-code/9/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T17:58:31.556 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T17:58:31.556 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T17:58:31.556 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T17:58:31.571 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:31.569+0000 7f3d67bf2900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:31.572 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:31.570+0000 7f3d67bf2900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:31.573 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:31.571+0000 7f3d67bf2900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:31.691 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:31.688+0000 7f3d67bf2900 -1 Falling back to public interface 2026-02-20T17:58:31.780 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 9 2026-02-20T17:58:31.780 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T17:58:31.780 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=9 2026-02-20T17:58:31.780 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T17:58:31.780 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T17:58:31.780 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:58:31.781 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T17:58:31.781 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T17:58:31.781 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:58:31.781 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-20T17:58:32.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T17:58:32.032 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:32.030+0000 7f3d67bf2900 -1 osd.9 0 log_to_monitors true 2026-02-20T17:58:33.005 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T17:58:33.006 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:58:33.006 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T17:58:33.006 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T17:58:33.007 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:58:33.007 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-20T17:58:33.229 INFO:tasks.workunit.client.0.trial195.stdout:osd.9 up in weight 1 up_from 46 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6874/2645394673,v1:127.0.0.1:6875/2645394673] [v2:127.0.0.1:6876/2645394673,v1:127.0.0.1:6877/2645394673] exists,up c0629070-187c-4594-aae4-4fd19faed37e 2026-02-20T17:58:33.233 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T17:58:33.233 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T17:58:33.233 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T17:58:33.233 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-20T17:58:33.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 10 2026-02-20T17:58:33.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-20T17:58:33.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T17:58:33.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=10 2026-02-20T17:58:33.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T17:58:33.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/10 2026-02-20T17:58:33.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-20T17:58:33.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T17:58:33.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T17:58:33.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T17:58:33.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/10' 2026-02-20T17:58:33.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/10/journal' 2026-02-20T17:58:33.235 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T17:58:33.235 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T17:58:33.235 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-20T17:58:33.235 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T17:58:33.235 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:58:33.235 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:58:33.235 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:58:33.235 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:33.235 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.58348 2026-02-20T17:58:33.235 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.58348/$cluster-$name.asok' 2026-02-20T17:58:33.235 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' 2026-02-20T17:58:33.235 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T17:58:33.236 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T17:58:33.236 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T17:58:33.236 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-20T17:58:33.236 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-20T17:58:33.236 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T17:58:33.236 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T17:58:33.236 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T17:58:33.236 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T17:58:33.236 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T17:58:33.236 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T17:58:33.236 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/10 2026-02-20T17:58:33.236 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T17:58:33.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=95ae3209-508b-423b-b3a6-9ed48c24e0e4 2026-02-20T17:58:33.237 INFO:tasks.workunit.client.0.trial195.stdout:add osd10 95ae3209-508b-423b-b3a6-9ed48c24e0e4 2026-02-20T17:58:33.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd10 95ae3209-508b-423b-b3a6-9ed48c24e0e4' 2026-02-20T17:58:33.237 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T17:58:33.246 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDJoJhp+oiRDhAAdGoHYbynFPDCEAoQ5rYNmw== 2026-02-20T17:58:33.246 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDJoJhp+oiRDhAAdGoHYbynFPDCEAoQ5rYNmw=="}' 2026-02-20T17:58:33.246 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 95ae3209-508b-423b-b3a6-9ed48c24e0e4 -i td/test-erasure-code/10/new.json 2026-02-20T17:58:33.477 INFO:tasks.workunit.client.0.trial195.stdout:10 2026-02-20T17:58:33.484 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/10/new.json 2026-02-20T17:58:33.485 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 10 --fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/10 --osd-journal=td/test-erasure-code/10/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDJoJhp+oiRDhAAdGoHYbynFPDCEAoQ5rYNmw== --osd-uuid 95ae3209-508b-423b-b3a6-9ed48c24e0e4 2026-02-20T17:58:33.503 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:33.501+0000 7fe55c267900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:33.505 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:33.503+0000 7fe55c267900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:33.506 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:33.504+0000 7fe55c267900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:33.506 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:33.504+0000 7fe55c267900 -1 bdev(0x558125970c00 td/test-erasure-code/10/block) open stat got: (1) Operation not permitted 2026-02-20T17:58:33.506 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:33.504+0000 7fe55c267900 -1 bluestore(td/test-erasure-code/10) _read_fsid unparsable uuid 2026-02-20T17:58:33.841 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/10/keyring 2026-02-20T17:58:33.841 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T17:58:33.842 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd10 key to auth repository 2026-02-20T17:58:33.842 INFO:tasks.workunit.client.0.trial195.stdout:adding osd10 key to auth repository 2026-02-20T17:58:33.843 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/10/keyring auth add osd.10 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T17:58:34.156 INFO:tasks.workunit.client.0.trial195.stdout:start osd.10 2026-02-20T17:58:34.158 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.10 2026-02-20T17:58:34.158 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 10 --fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/10 --osd-journal=td/test-erasure-code/10/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T17:58:34.158 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T17:58:34.158 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T17:58:34.158 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T17:58:34.173 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:34.171+0000 7fe450956900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:34.174 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:34.172+0000 7fe450956900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:34.175 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:34.173+0000 7fe450956900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:34.312 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:34.310+0000 7fe450956900 -1 Falling back to public interface 2026-02-20T17:58:34.379 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 10 2026-02-20T17:58:34.379 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T17:58:34.380 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=10 2026-02-20T17:58:34.380 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T17:58:34.380 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T17:58:34.380 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:58:34.380 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T17:58:34.380 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T17:58:34.380 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:58:34.380 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.10 up' 2026-02-20T17:58:34.605 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T17:58:34.662 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:34.660+0000 7fe450956900 -1 osd.10 0 log_to_monitors true 2026-02-20T17:58:35.606 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T17:58:35.607 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:58:35.607 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T17:58:35.607 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T17:58:35.608 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:58:35.608 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.10 up' 2026-02-20T17:58:35.831 INFO:tasks.workunit.client.0.trial195.stdout:osd.10 up in weight 1 up_from 50 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6882/1682604285,v1:127.0.0.1:6883/1682604285] [v2:127.0.0.1:6884/1682604285,v1:127.0.0.1:6885/1682604285] exists,up 95ae3209-508b-423b-b3a6-9ed48c24e0e4 2026-02-20T17:58:35.832 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T17:58:35.832 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T17:58:35.832 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T17:58:35.832 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:39: run: create_rbd_pool 2026-02-20T17:58:35.832 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-20T17:58:36.052 INFO:tasks.workunit.client.0.trial195.stderr:pool 'rbd' does not exist 2026-02-20T17:58:36.061 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-20T17:58:36.062 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-20T17:58:36.335 INFO:tasks.workunit.client.0.trial195.stderr:pool 'rbd' already exists 2026-02-20T17:58:36.343 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-20T17:58:37.344 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-20T17:58:37.850 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:40: run: wait_for_clean 2026-02-20T17:58:37.850 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T17:58:37.850 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T17:58:37.850 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T17:58:37.850 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T17:58:37.851 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T17:58:37.851 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T17:58:37.851 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T17:58:37.852 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T17:58:37.852 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T17:58:37.932 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T17:58:37.932 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T17:58:37.932 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T17:58:37.932 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T17:58:37.932 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T17:58:37.932 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T17:58:38.154 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T17:58:38.154 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T17:58:38.154 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T17:58:38.154 INFO:tasks.workunit.client.0.trial195.stderr:3 2026-02-20T17:58:38.154 INFO:tasks.workunit.client.0.trial195.stderr:4 2026-02-20T17:58:38.154 INFO:tasks.workunit.client.0.trial195.stderr:5 2026-02-20T17:58:38.154 INFO:tasks.workunit.client.0.trial195.stderr:6 2026-02-20T17:58:38.154 INFO:tasks.workunit.client.0.trial195.stderr:7 2026-02-20T17:58:38.155 INFO:tasks.workunit.client.0.trial195.stderr:8 2026-02-20T17:58:38.155 INFO:tasks.workunit.client.0.trial195.stderr:9 2026-02-20T17:58:38.155 INFO:tasks.workunit.client.0.trial195.stderr:10' 2026-02-20T17:58:38.155 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T17:58:38.155 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:58:38.155 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T17:58:38.216 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836488 2026-02-20T17:58:38.216 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836488 2026-02-20T17:58:38.216 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836488' 2026-02-20T17:58:38.217 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:58:38.217 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T17:58:38.279 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672969 2026-02-20T17:58:38.279 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672969 2026-02-20T17:58:38.279 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836488 1-42949672969' 2026-02-20T17:58:38.279 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:58:38.279 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T17:58:38.344 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509447 2026-02-20T17:58:38.344 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509447 2026-02-20T17:58:38.344 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836488 1-42949672969 2-64424509447' 2026-02-20T17:58:38.344 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:58:38.344 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T17:58:38.409 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378631 2026-02-20T17:58:38.410 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378631 2026-02-20T17:58:38.410 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836488 1-42949672969 2-64424509447 3-81604378631' 2026-02-20T17:58:38.410 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:58:38.410 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-20T17:58:38.474 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=103079215110 2026-02-20T17:58:38.474 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 103079215110 2026-02-20T17:58:38.474 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836488 1-42949672969 2-64424509447 3-81604378631 4-103079215110' 2026-02-20T17:58:38.474 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:58:38.474 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-20T17:58:38.538 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=120259084295 2026-02-20T17:58:38.538 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 120259084295 2026-02-20T17:58:38.538 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836488 1-42949672969 2-64424509447 3-81604378631 4-103079215110 5-120259084295' 2026-02-20T17:58:38.538 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:58:38.538 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-20T17:58:38.602 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920773 2026-02-20T17:58:38.602 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920773 2026-02-20T17:58:38.602 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836488 1-42949672969 2-64424509447 3-81604378631 4-103079215110 5-120259084295 6-141733920773' 2026-02-20T17:58:38.602 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:58:38.602 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-20T17:58:38.665 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757253 2026-02-20T17:58:38.665 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757253 2026-02-20T17:58:38.665 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836488 1-42949672969 2-64424509447 3-81604378631 4-103079215110 5-120259084295 6-141733920773 7-163208757253' 2026-02-20T17:58:38.665 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:58:38.666 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-20T17:58:38.731 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=180388626436 2026-02-20T17:58:38.731 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 180388626436 2026-02-20T17:58:38.731 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836488 1-42949672969 2-64424509447 3-81604378631 4-103079215110 5-120259084295 6-141733920773 7-163208757253 8-180388626436' 2026-02-20T17:58:38.731 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:58:38.731 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-20T17:58:38.795 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=197568495621 2026-02-20T17:58:38.795 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 197568495621 2026-02-20T17:58:38.795 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836488 1-42949672969 2-64424509447 3-81604378631 4-103079215110 5-120259084295 6-141733920773 7-163208757253 8-180388626436 9-197568495621' 2026-02-20T17:58:38.795 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:58:38.795 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.10 flush_pg_stats 2026-02-20T17:58:38.860 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=214748364803 2026-02-20T17:58:38.860 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 214748364803 2026-02-20T17:58:38.860 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836488 1-42949672969 2-64424509447 3-81604378631 4-103079215110 5-120259084295 6-141733920773 7-163208757253 8-180388626436 9-197568495621 10-214748364803' 2026-02-20T17:58:38.860 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:58:38.861 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836488 2026-02-20T17:58:38.861 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:58:38.862 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T17:58:38.862 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836488 2026-02-20T17:58:38.862 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:58:38.863 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836488 2026-02-20T17:58:38.863 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836488' 2026-02-20T17:58:38.863 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.0 seq 21474836488 2026-02-20T17:58:38.864 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T17:58:39.092 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836486 -lt 21474836488 2026-02-20T17:58:39.092 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T17:58:40.093 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T17:58:40.093 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T17:58:40.323 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836488 -lt 21474836488 2026-02-20T17:58:40.323 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:58:40.323 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672969 2026-02-20T17:58:40.324 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:58:40.324 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T17:58:40.325 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672969 2026-02-20T17:58:40.325 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:58:40.326 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672969 2026-02-20T17:58:40.326 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672969' 2026-02-20T17:58:40.327 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.1 seq 42949672969 2026-02-20T17:58:40.327 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T17:58:40.551 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672969 -lt 42949672969 2026-02-20T17:58:40.551 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:58:40.552 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509447 2026-02-20T17:58:40.552 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:58:40.552 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T17:58:40.553 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509447 2026-02-20T17:58:40.553 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:58:40.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509447 2026-02-20T17:58:40.554 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.2 seq 64424509447 2026-02-20T17:58:40.555 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509447' 2026-02-20T17:58:40.555 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T17:58:40.779 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509447 -lt 64424509447 2026-02-20T17:58:40.779 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:58:40.780 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-81604378631 2026-02-20T17:58:40.780 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:58:40.780 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T17:58:40.781 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-81604378631 2026-02-20T17:58:40.781 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:58:40.782 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=81604378631 2026-02-20T17:58:40.783 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.3 seq 81604378631 2026-02-20T17:58:40.783 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 81604378631' 2026-02-20T17:58:40.783 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T17:58:41.013 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 81604378631 -lt 81604378631 2026-02-20T17:58:41.014 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:58:41.014 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-103079215110 2026-02-20T17:58:41.014 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:58:41.015 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-20T17:58:41.016 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-103079215110 2026-02-20T17:58:41.016 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:58:41.017 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=103079215110 2026-02-20T17:58:41.017 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 103079215110' 2026-02-20T17:58:41.017 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.4 seq 103079215110 2026-02-20T17:58:41.018 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-20T17:58:41.240 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 103079215110 -lt 103079215110 2026-02-20T17:58:41.240 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:58:41.241 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-120259084295 2026-02-20T17:58:41.241 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:58:41.242 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-20T17:58:41.242 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-120259084295 2026-02-20T17:58:41.242 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:58:41.243 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=120259084295 2026-02-20T17:58:41.244 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.5 seq 120259084295 2026-02-20T17:58:41.244 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 120259084295' 2026-02-20T17:58:41.244 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-20T17:58:41.471 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 120259084295 -lt 120259084295 2026-02-20T17:58:41.471 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:58:41.472 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-141733920773 2026-02-20T17:58:41.472 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:58:41.472 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-20T17:58:41.473 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-141733920773 2026-02-20T17:58:41.473 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:58:41.474 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=141733920773 2026-02-20T17:58:41.474 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 141733920773' 2026-02-20T17:58:41.475 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.6 seq 141733920773 2026-02-20T17:58:41.475 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-20T17:58:41.702 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 141733920773 -lt 141733920773 2026-02-20T17:58:41.702 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:58:41.703 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-163208757253 2026-02-20T17:58:41.703 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:58:41.704 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-20T17:58:41.704 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-163208757253 2026-02-20T17:58:41.705 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:58:41.706 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=163208757253 2026-02-20T17:58:41.706 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 163208757253' 2026-02-20T17:58:41.706 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.7 seq 163208757253 2026-02-20T17:58:41.706 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-20T17:58:41.928 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 163208757253 -lt 163208757253 2026-02-20T17:58:41.929 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:58:41.929 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-180388626436 2026-02-20T17:58:41.929 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:58:41.930 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-20T17:58:41.930 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-180388626436 2026-02-20T17:58:41.931 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:58:41.932 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=180388626436 2026-02-20T17:58:41.932 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 180388626436' 2026-02-20T17:58:41.932 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.8 seq 180388626436 2026-02-20T17:58:41.932 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-20T17:58:42.158 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 180388626437 -lt 180388626436 2026-02-20T17:58:42.158 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:58:42.159 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-197568495621 2026-02-20T17:58:42.159 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:58:42.160 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-20T17:58:42.161 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-197568495621 2026-02-20T17:58:42.161 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:58:42.161 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=197568495621 2026-02-20T17:58:42.162 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.9 seq 197568495621 2026-02-20T17:58:42.162 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 197568495621' 2026-02-20T17:58:42.162 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-20T17:58:42.391 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 197568495621 -lt 197568495621 2026-02-20T17:58:42.391 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:58:42.392 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-214748364803 2026-02-20T17:58:42.392 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:58:42.392 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=10 2026-02-20T17:58:42.393 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 10-214748364803 2026-02-20T17:58:42.393 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:58:42.394 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=214748364803 2026-02-20T17:58:42.394 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.10 seq 214748364803 2026-02-20T17:58:42.395 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.10 seq 214748364803' 2026-02-20T17:58:42.395 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-20T17:58:42.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 214748364804 -lt 214748364803 2026-02-20T17:58:42.620 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T17:58:42.621 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T17:58:42.621 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T17:58:42.939 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-20T17:58:42.939 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T17:58:42.939 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T17:58:42.939 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T17:58:42.940 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T17:58:42.940 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T17:58:42.940 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T17:58:42.940 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T17:58:43.166 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-20T17:58:43.167 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T17:58:43.167 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T17:58:43.167 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T17:58:43.482 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-20T17:58:43.482 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T17:58:43.482 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T17:58:43.483 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:42: run: get_asok_path osd.0 2026-02-20T17:58:43.483 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-20T17:58:43.483 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-20T17:58:43.484 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T17:58:43.484 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:43.484 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.58348 2026-02-20T17:58:43.484 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.58348/ceph-osd.0.asok 2026-02-20T17:58:43.484 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:42: run: CEPH_ARGS= 2026-02-20T17:58:43.484 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:42: run: ceph --admin-daemon /tmp/ceph-asok.58348/ceph-osd.0.asok log flush 2026-02-20T17:58:43.528 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T17:58:43.533 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:43: run: grep 'load: jerasure.*lrc' td/test-erasure-code/osd.0.log 2026-02-20T17:58:43.534 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T17:58:08.868+0000 7f79bc070900 0 load: jerasure load: lrc 2026-02-20T17:58:43.535 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:44: run: create_erasure_coded_pool ecpool 2026-02-20T17:58:43.535 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:56: create_erasure_coded_pool: local poolname=ecpool 2026-02-20T17:58:43.535 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:58: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd 2026-02-20T17:58:43.819 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:60: create_erasure_coded_pool: create_pool ecpool 12 12 erasure myprofile 2026-02-20T17:58:43.820 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create ecpool 12 12 erasure myprofile 2026-02-20T17:58:44.216 INFO:tasks.workunit.client.0.trial195.stderr:pool 'ecpool' already exists 2026-02-20T17:58:44.225 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-20T17:58:45.226 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:62: create_erasure_coded_pool: wait_for_clean 2026-02-20T17:58:45.226 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T17:58:45.226 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T17:58:45.226 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T17:58:45.226 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T17:58:45.227 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T17:58:45.227 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T17:58:45.227 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T17:58:45.227 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T17:58:45.227 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T17:58:45.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T17:58:45.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T17:58:45.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T17:58:45.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T17:58:45.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T17:58:45.307 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T17:58:45.531 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T17:58:45.532 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T17:58:45.532 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T17:58:45.532 INFO:tasks.workunit.client.0.trial195.stderr:3 2026-02-20T17:58:45.532 INFO:tasks.workunit.client.0.trial195.stderr:4 2026-02-20T17:58:45.532 INFO:tasks.workunit.client.0.trial195.stderr:5 2026-02-20T17:58:45.532 INFO:tasks.workunit.client.0.trial195.stderr:6 2026-02-20T17:58:45.532 INFO:tasks.workunit.client.0.trial195.stderr:7 2026-02-20T17:58:45.532 INFO:tasks.workunit.client.0.trial195.stderr:8 2026-02-20T17:58:45.532 INFO:tasks.workunit.client.0.trial195.stderr:9 2026-02-20T17:58:45.533 INFO:tasks.workunit.client.0.trial195.stderr:10' 2026-02-20T17:58:45.533 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T17:58:45.533 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:58:45.533 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T17:58:45.596 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836492 2026-02-20T17:58:45.596 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836492 2026-02-20T17:58:45.596 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836492' 2026-02-20T17:58:45.596 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:58:45.597 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T17:58:45.661 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672972 2026-02-20T17:58:45.661 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672972 2026-02-20T17:58:45.662 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836492 1-42949672972' 2026-02-20T17:58:45.662 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:58:45.662 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T17:58:45.725 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509451 2026-02-20T17:58:45.725 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509451 2026-02-20T17:58:45.725 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836492 1-42949672972 2-64424509451' 2026-02-20T17:58:45.725 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:58:45.725 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T17:58:45.794 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378634 2026-02-20T17:58:45.794 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378634 2026-02-20T17:58:45.794 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836492 1-42949672972 2-64424509451 3-81604378634' 2026-02-20T17:58:45.794 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:58:45.794 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-20T17:58:45.856 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=103079215114 2026-02-20T17:58:45.857 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 103079215114 2026-02-20T17:58:45.857 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836492 1-42949672972 2-64424509451 3-81604378634 4-103079215114' 2026-02-20T17:58:45.857 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:58:45.857 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-20T17:58:45.921 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=120259084298 2026-02-20T17:58:45.921 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 120259084298 2026-02-20T17:58:45.921 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836492 1-42949672972 2-64424509451 3-81604378634 4-103079215114 5-120259084298' 2026-02-20T17:58:45.921 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:58:45.921 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-20T17:58:45.987 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920777 2026-02-20T17:58:45.987 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920777 2026-02-20T17:58:45.987 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836492 1-42949672972 2-64424509451 3-81604378634 4-103079215114 5-120259084298 6-141733920777' 2026-02-20T17:58:45.988 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:58:45.988 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-20T17:58:46.054 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757256 2026-02-20T17:58:46.054 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757256 2026-02-20T17:58:46.054 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836492 1-42949672972 2-64424509451 3-81604378634 4-103079215114 5-120259084298 6-141733920777 7-163208757256' 2026-02-20T17:58:46.054 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:58:46.054 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-20T17:58:46.117 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=180388626440 2026-02-20T17:58:46.117 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 180388626440 2026-02-20T17:58:46.117 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836492 1-42949672972 2-64424509451 3-81604378634 4-103079215114 5-120259084298 6-141733920777 7-163208757256 8-180388626440' 2026-02-20T17:58:46.117 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:58:46.117 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-20T17:58:46.179 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=197568495624 2026-02-20T17:58:46.180 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 197568495624 2026-02-20T17:58:46.180 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836492 1-42949672972 2-64424509451 3-81604378634 4-103079215114 5-120259084298 6-141733920777 7-163208757256 8-180388626440 9-197568495624' 2026-02-20T17:58:46.180 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:58:46.180 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.10 flush_pg_stats 2026-02-20T17:58:46.243 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=214748364807 2026-02-20T17:58:46.244 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 214748364807 2026-02-20T17:58:46.244 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836492 1-42949672972 2-64424509451 3-81604378634 4-103079215114 5-120259084298 6-141733920777 7-163208757256 8-180388626440 9-197568495624 10-214748364807' 2026-02-20T17:58:46.244 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:58:46.244 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836492 2026-02-20T17:58:46.245 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:58:46.245 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T17:58:46.246 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836492 2026-02-20T17:58:46.246 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:58:46.247 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836492 2026-02-20T17:58:46.247 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.0 seq 21474836492 2026-02-20T17:58:46.247 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836492' 2026-02-20T17:58:46.248 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T17:58:46.471 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836492 -lt 21474836492 2026-02-20T17:58:46.471 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:58:46.471 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672972 2026-02-20T17:58:46.472 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:58:46.472 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T17:58:46.473 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672972 2026-02-20T17:58:46.473 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:58:46.474 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672972 2026-02-20T17:58:46.474 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.1 seq 42949672972 2026-02-20T17:58:46.475 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672972' 2026-02-20T17:58:46.475 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T17:58:46.703 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672972 -lt 42949672972 2026-02-20T17:58:46.703 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:58:46.703 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509451 2026-02-20T17:58:46.704 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:58:46.704 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T17:58:46.705 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509451 2026-02-20T17:58:46.705 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:58:46.706 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509451 2026-02-20T17:58:46.707 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509451' 2026-02-20T17:58:46.707 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.2 seq 64424509451 2026-02-20T17:58:46.707 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T17:58:46.931 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509451 -lt 64424509451 2026-02-20T17:58:46.931 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:58:46.932 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-81604378634 2026-02-20T17:58:46.932 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:58:46.932 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T17:58:46.933 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-81604378634 2026-02-20T17:58:46.933 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:58:46.934 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=81604378634 2026-02-20T17:58:46.934 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.3 seq 81604378634 2026-02-20T17:58:46.935 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 81604378634' 2026-02-20T17:58:46.935 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T17:58:47.160 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 81604378634 -lt 81604378634 2026-02-20T17:58:47.160 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:58:47.161 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-103079215114 2026-02-20T17:58:47.161 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:58:47.162 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-20T17:58:47.162 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-103079215114 2026-02-20T17:58:47.162 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:58:47.164 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=103079215114 2026-02-20T17:58:47.164 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 103079215114' 2026-02-20T17:58:47.164 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.4 seq 103079215114 2026-02-20T17:58:47.164 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-20T17:58:47.391 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 103079215114 -lt 103079215114 2026-02-20T17:58:47.391 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:58:47.391 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-120259084298 2026-02-20T17:58:47.391 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:58:47.392 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-20T17:58:47.393 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-120259084298 2026-02-20T17:58:47.393 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:58:47.394 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=120259084298 2026-02-20T17:58:47.394 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.5 seq 120259084298 2026-02-20T17:58:47.395 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 120259084298' 2026-02-20T17:58:47.395 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-20T17:58:47.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 120259084296 -lt 120259084298 2026-02-20T17:58:47.621 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T17:58:48.621 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T17:58:48.622 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-20T17:58:48.845 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 120259084298 -lt 120259084298 2026-02-20T17:58:48.845 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:58:48.846 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-141733920777 2026-02-20T17:58:48.846 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:58:48.846 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-20T17:58:48.847 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-141733920777 2026-02-20T17:58:48.847 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:58:48.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=141733920777 2026-02-20T17:58:48.848 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.6 seq 141733920777 2026-02-20T17:58:48.849 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 141733920777' 2026-02-20T17:58:48.849 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-20T17:58:49.071 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 141733920777 -lt 141733920777 2026-02-20T17:58:49.071 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:58:49.072 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-163208757256 2026-02-20T17:58:49.072 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:58:49.073 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-20T17:58:49.073 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-163208757256 2026-02-20T17:58:49.074 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:58:49.074 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=163208757256 2026-02-20T17:58:49.075 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.7 seq 163208757256 2026-02-20T17:58:49.075 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 163208757256' 2026-02-20T17:58:49.075 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-20T17:58:49.301 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 163208757256 -lt 163208757256 2026-02-20T17:58:49.301 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:58:49.301 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-180388626440 2026-02-20T17:58:49.302 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:58:49.302 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-20T17:58:49.303 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-180388626440 2026-02-20T17:58:49.303 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:58:49.304 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=180388626440 2026-02-20T17:58:49.304 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 180388626440' 2026-02-20T17:58:49.304 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.8 seq 180388626440 2026-02-20T17:58:49.305 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-20T17:58:49.530 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 180388626440 -lt 180388626440 2026-02-20T17:58:49.530 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:58:49.530 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-197568495624 2026-02-20T17:58:49.531 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:58:49.531 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-20T17:58:49.532 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-197568495624 2026-02-20T17:58:49.532 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:58:49.533 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=197568495624 2026-02-20T17:58:49.533 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 197568495624' 2026-02-20T17:58:49.534 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.9 seq 197568495624 2026-02-20T17:58:49.534 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-20T17:58:49.757 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 197568495624 -lt 197568495624 2026-02-20T17:58:49.757 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:58:49.758 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-214748364807 2026-02-20T17:58:49.758 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:58:49.758 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=10 2026-02-20T17:58:49.759 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 10-214748364807 2026-02-20T17:58:49.759 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:58:49.760 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=214748364807 2026-02-20T17:58:49.761 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.10 seq 214748364807 2026-02-20T17:58:49.761 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.10 seq 214748364807' 2026-02-20T17:58:49.761 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-20T17:58:49.987 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 214748364807 -lt 214748364807 2026-02-20T17:58:49.987 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T17:58:49.988 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T17:58:49.988 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T17:58:50.298 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 16 == 0 2026-02-20T17:58:50.298 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T17:58:50.299 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T17:58:50.299 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T17:58:50.299 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T17:58:50.299 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T17:58:50.300 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T17:58:50.300 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T17:58:50.519 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=16 2026-02-20T17:58:50.520 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T17:58:50.520 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T17:58:50.521 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T17:58:50.837 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 = 16 2026-02-20T17:58:50.838 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T17:58:50.838 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T17:58:50.838 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:46: run: set 2026-02-20T17:58:50.839 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:46: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-20T17:58:50.840 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:46: run: local 'funcs=TEST_alignment_constraints 2026-02-20T17:58:50.840 INFO:tasks.workunit.client.0.trial195.stderr:TEST_chunk_mapping 2026-02-20T17:58:50.840 INFO:tasks.workunit.client.0.trial195.stderr:TEST_rados_put_get_isa 2026-02-20T17:58:50.840 INFO:tasks.workunit.client.0.trial195.stderr:TEST_rados_put_get_jerasure 2026-02-20T17:58:50.840 INFO:tasks.workunit.client.0.trial195.stderr:TEST_rados_put_get_lrc_advanced 2026-02-20T17:58:50.841 INFO:tasks.workunit.client.0.trial195.stderr:TEST_rados_put_get_lrc_kml 2026-02-20T17:58:50.841 INFO:tasks.workunit.client.0.trial195.stderr:TEST_rados_put_get_shec' 2026-02-20T17:58:50.841 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-20T17:58:50.841 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_alignment_constraints td/test-erasure-code 2026-02-20T17:58:50.841 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: TEST_alignment_constraints: local payload=ABC 2026-02-20T17:58:50.841 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:254: TEST_alignment_constraints: echo ABC 2026-02-20T17:58:50.841 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:260: TEST_alignment_constraints: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-20T17:58:50.851 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:260: TEST_alignment_constraints: local stripe_unit=4096 2026-02-20T17:58:50.852 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:261: TEST_alignment_constraints: ceph osd erasure-code-profile get myprofile 2026-02-20T17:58:50.852 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:261: TEST_alignment_constraints: grep k= 2026-02-20T17:58:51.087 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:261: TEST_alignment_constraints: eval local k=2 2026-02-20T17:58:51.087 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:261: TEST_alignment_constraints: local k=2 2026-02-20T17:58:51.088 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:262: TEST_alignment_constraints: local block_size=8191 2026-02-20T17:58:51.088 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:263: TEST_alignment_constraints: dd if=/dev/zero of=td/test-erasure-code/ORIGINAL bs=8191 count=2 2026-02-20T17:58:51.088 INFO:tasks.workunit.client.0.trial195.stderr:2+0 records in 2026-02-20T17:58:51.089 INFO:tasks.workunit.client.0.trial195.stderr:2+0 records out 2026-02-20T17:58:51.089 INFO:tasks.workunit.client.0.trial195.stderr:16382 bytes (16 kB, 16 KiB) copied, 8.573e-05 s, 191 MB/s 2026-02-20T17:58:51.089 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:264: TEST_alignment_constraints: rados --block-size=8191 --pool ecpool put UNALIGNED td/test-erasure-code/ORIGINAL 2026-02-20T17:58:51.106 INFO:tasks.workunit.client.0.trial195.stderr:INFO: op_size has been rounded to 8192 2026-02-20T17:58:51.110 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:266: TEST_alignment_constraints: rm td/test-erasure-code/ORIGINAL 2026-02-20T17:58:51.111 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-20T17:58:51.111 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_chunk_mapping td/test-erasure-code 2026-02-20T17:58:51.111 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:304: TEST_chunk_mapping: local dir=td/test-erasure-code 2026-02-20T17:58:51.111 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:311: TEST_chunk_mapping: verify_chunk_mapping td/test-erasure-code ecpool 0 1 2026-02-20T17:58:51.111 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:281: verify_chunk_mapping: local dir=td/test-erasure-code 2026-02-20T17:58:51.112 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:282: verify_chunk_mapping: local poolname=ecpool 2026-02-20T17:58:51.112 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:283: verify_chunk_mapping: local first=0 2026-02-20T17:58:51.112 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:284: verify_chunk_mapping: local second=1 2026-02-20T17:58:51.112 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:286: verify_chunk_mapping: chunk_size 2026-02-20T17:58:51.112 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:270: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-20T17:58:51.122 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:270: chunk_size: echo 4096 2026-02-20T17:58:51.122 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:286: verify_chunk_mapping: printf '%*s' 4096 FIRSTecpool 2026-02-20T17:58:51.123 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:286: verify_chunk_mapping: chunk_size 2026-02-20T17:58:51.123 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:270: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-20T17:58:51.133 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:270: chunk_size: echo 4096 2026-02-20T17:58:51.133 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:286: verify_chunk_mapping: printf '%*s' 4096 SECONDecpool 2026-02-20T17:58:51.135 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:286: verify_chunk_mapping: local 'payload= FIRSTecpool SECONDecpool' 2026-02-20T17:58:51.135 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:287: verify_chunk_mapping: echo -n ' FIRSTecpool SECONDecpool' 2026-02-20T17:58:51.136 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:289: verify_chunk_mapping: rados --pool ecpool put SOMETHINGecpool td/test-erasure-code/ORIGINAL 2026-02-20T17:58:51.154 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:290: verify_chunk_mapping: rados --pool ecpool get SOMETHINGecpool td/test-erasure-code/COPY 2026-02-20T17:58:51.176 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:291: verify_chunk_mapping: get_osds ecpool SOMETHINGecpool 2026-02-20T17:58:51.176 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-20T17:58:51.176 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHINGecpool 2026-02-20T17:58:51.176 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool SOMETHINGecpool 2026-02-20T17:58:51.176 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T17:58:51.403 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T17:58:51.403 INFO:tasks.workunit.client.0.trial195.stderr:5 2026-02-20T17:58:51.403 INFO:tasks.workunit.client.0.trial195.stderr:9 2026-02-20T17:58:51.403 INFO:tasks.workunit.client.0.trial195.stderr:0' 2026-02-20T17:58:51.403 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 9 0 2026-02-20T17:58:51.404 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:291: verify_chunk_mapping: osds=('3' '5' '9' '0') 2026-02-20T17:58:51.404 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:291: verify_chunk_mapping: local -a osds 2026-02-20T17:58:51.404 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i = 0 )) 2026-02-20T17:58:51.404 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i < 4 )) 2026-02-20T17:58:51.404 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:293: verify_chunk_mapping: ceph daemon osd.3 flush_journal 2026-02-20T17:58:51.461 INFO:tasks.workunit.client.0.trial195.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-20T17:58:51.463 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i++ )) 2026-02-20T17:58:51.463 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i < 4 )) 2026-02-20T17:58:51.463 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:293: verify_chunk_mapping: ceph daemon osd.5 flush_journal 2026-02-20T17:58:51.519 INFO:tasks.workunit.client.0.trial195.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-20T17:58:51.521 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i++ )) 2026-02-20T17:58:51.521 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i < 4 )) 2026-02-20T17:58:51.521 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:293: verify_chunk_mapping: ceph daemon osd.9 flush_journal 2026-02-20T17:58:51.578 INFO:tasks.workunit.client.0.trial195.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-20T17:58:51.579 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i++ )) 2026-02-20T17:58:51.579 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i < 4 )) 2026-02-20T17:58:51.580 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:293: verify_chunk_mapping: ceph daemon osd.0 flush_journal 2026-02-20T17:58:51.636 INFO:tasks.workunit.client.0.trial195.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-20T17:58:51.638 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i++ )) 2026-02-20T17:58:51.638 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i < 4 )) 2026-02-20T17:58:51.638 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-20T17:58:51.638 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:296: verify_chunk_mapping: rm td/test-erasure-code/COPY 2026-02-20T17:58:51.640 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:298: verify_chunk_mapping: get_osds ecpool SOMETHINGecpool 2026-02-20T17:58:51.640 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-20T17:58:51.640 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHINGecpool 2026-02-20T17:58:51.640 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool SOMETHINGecpool 2026-02-20T17:58:51.640 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T17:58:51.866 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T17:58:51.866 INFO:tasks.workunit.client.0.trial195.stderr:5 2026-02-20T17:58:51.866 INFO:tasks.workunit.client.0.trial195.stderr:9 2026-02-20T17:58:51.867 INFO:tasks.workunit.client.0.trial195.stderr:0' 2026-02-20T17:58:51.867 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 9 0 2026-02-20T17:58:51.867 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:298: verify_chunk_mapping: osds=('3' '5' '9' '0') 2026-02-20T17:58:51.867 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:298: verify_chunk_mapping: local -a osds 2026-02-20T17:58:51.868 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:299: verify_chunk_mapping: objectstore_tool td/test-erasure-code 3 SOMETHINGecpool get-bytes 2026-02-20T17:58:51.868 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-code 2026-02-20T17:58:51.868 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-20T17:58:51.868 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-20T17:58:51.868 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:299: verify_chunk_mapping: grep --quiet FIRSTecpool 2026-02-20T17:58:51.868 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-20T17:58:51.868 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-code 3 SOMETHINGecpool get-bytes 2026-02-20T17:58:51.868 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-code 2026-02-20T17:58:51.868 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-20T17:58:51.868 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-20T17:58:51.868 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-20T17:58:51.868 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-code TERM osd.3 2026-02-20T17:58:51.868 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T17:58:51.868 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T17:58:51.868 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T17:58:51.869 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T17:58:51.869 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T17:58:52.174 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T17:58:52.174 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-code 3 SOMETHINGecpool get-bytes 2026-02-20T17:58:52.174 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-code 2026-02-20T17:58:52.175 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-20T17:58:52.175 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-20T17:58:52.175 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-20T17:58:52.175 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-code/3 2026-02-20T17:58:52.175 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-code/3 SOMETHINGecpool get-bytes 2026-02-20T17:58:52.259 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-code 3 2026-02-20T17:58:52.259 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-code 2026-02-20T17:58:52.259 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-20T17:58:52.259 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-20T17:58:52.259 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-20T17:58:52.259 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-code/3 2026-02-20T17:58:52.259 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-20T17:58:52.260 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T17:58:52.260 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T17:58:52.260 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T17:58:52.260 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-code/3' 2026-02-20T17:58:52.260 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-code/3/journal' 2026-02-20T17:58:52.260 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-20T17:58:52.260 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-20T17:58:52.260 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-20T17:58:52.260 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-20T17:58:52.260 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:58:52.260 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:58:52.260 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:58:52.260 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:52.261 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.58348 2026-02-20T17:58:52.261 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.58348/$cluster-$name.asok' 2026-02-20T17:58:52.261 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' 2026-02-20T17:58:52.261 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-20T17:58:52.261 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-20T17:58:52.261 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-20T17:58:52.261 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T17:58:52.261 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T17:58:52.261 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T17:58:52.261 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T17:58:52.261 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-20T17:58:52.261 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-20T17:58:52.262 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-code/3 2026-02-20T17:58:52.262 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-20T17:58:52.262 INFO:tasks.workunit.client.0.trial195.stderr:start osd.3 2026-02-20T17:58:52.262 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/3 --osd-journal=td/test-erasure-code/3/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T17:58:52.262 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-code/3/whoami 2026-02-20T17:58:52.263 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-20T17:58:52.263 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-20T17:58:52.263 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-20T17:58:52.263 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-20T17:58:52.274 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:52.272+0000 7f0b3692d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:52.275 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:52.273+0000 7f0b3692d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:52.276 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:52.274+0000 7f0b3692d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:52.415 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:52.413+0000 7f0b3692d900 -1 Falling back to public interface 2026-02-20T17:58:52.484 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-20T17:58:52.485 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T17:58:52.485 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-20T17:58:52.485 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T17:58:52.485 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T17:58:52.485 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:58:52.485 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T17:58:52.485 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T17:58:52.485 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:58:52.485 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T17:58:52.706 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T17:58:52.763 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:52.761+0000 7f0b3692d900 -1 osd.3 70 log_to_monitors true 2026-02-20T17:58:53.707 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T17:58:53.707 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:58:53.707 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T17:58:53.707 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T17:58:53.708 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:58:53.708 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T17:58:53.931 INFO:tasks.workunit.client.0.trial195.stderr:osd.3 up in weight 1 up_from 74 up_thru 74 down_at 71 last_clean_interval [19,70) [v2:127.0.0.1:6826/3456956723,v1:127.0.0.1:6827/3456956723] [v2:127.0.0.1:6828/3456956723,v1:127.0.0.1:6829/3456956723] exists,up cecb896c-3ca9-406e-84e5-4e0aa187e970 2026-02-20T17:58:53.932 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T17:58:53.932 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T17:58:53.932 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T17:58:53.932 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-20T17:58:53.932 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T17:58:53.932 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T17:58:53.932 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T17:58:53.932 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T17:58:53.932 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T17:58:53.932 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T17:58:53.932 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T17:58:53.933 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T17:58:53.933 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T17:58:54.010 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T17:58:54.010 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T17:58:54.010 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T17:58:54.010 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T17:58:54.010 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T17:58:54.010 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T17:58:54.235 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T17:58:54.235 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T17:58:54.235 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T17:58:54.235 INFO:tasks.workunit.client.0.trial195.stderr:3 2026-02-20T17:58:54.235 INFO:tasks.workunit.client.0.trial195.stderr:4 2026-02-20T17:58:54.235 INFO:tasks.workunit.client.0.trial195.stderr:5 2026-02-20T17:58:54.235 INFO:tasks.workunit.client.0.trial195.stderr:6 2026-02-20T17:58:54.235 INFO:tasks.workunit.client.0.trial195.stderr:7 2026-02-20T17:58:54.236 INFO:tasks.workunit.client.0.trial195.stderr:8 2026-02-20T17:58:54.236 INFO:tasks.workunit.client.0.trial195.stderr:9 2026-02-20T17:58:54.236 INFO:tasks.workunit.client.0.trial195.stderr:10' 2026-02-20T17:58:54.236 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T17:58:54.236 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:58:54.236 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T17:58:54.300 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836495 2026-02-20T17:58:54.300 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836495 2026-02-20T17:58:54.300 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836495' 2026-02-20T17:58:54.300 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:58:54.300 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T17:58:54.364 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672976 2026-02-20T17:58:54.364 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672976 2026-02-20T17:58:54.364 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836495 1-42949672976' 2026-02-20T17:58:54.364 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:58:54.364 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T17:58:54.427 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509454 2026-02-20T17:58:54.427 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509454 2026-02-20T17:58:54.427 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836495 1-42949672976 2-64424509454' 2026-02-20T17:58:54.427 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:58:54.427 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T17:58:54.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=317827579907 2026-02-20T17:58:54.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 317827579907 2026-02-20T17:58:54.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836495 1-42949672976 2-64424509454 3-317827579907' 2026-02-20T17:58:54.493 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:58:54.493 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-20T17:58:54.558 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=103079215117 2026-02-20T17:58:54.558 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 103079215117 2026-02-20T17:58:54.558 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836495 1-42949672976 2-64424509454 3-317827579907 4-103079215117' 2026-02-20T17:58:54.558 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:58:54.559 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-20T17:58:54.623 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=120259084302 2026-02-20T17:58:54.623 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 120259084302 2026-02-20T17:58:54.623 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836495 1-42949672976 2-64424509454 3-317827579907 4-103079215117 5-120259084302' 2026-02-20T17:58:54.624 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:58:54.624 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-20T17:58:54.690 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920780 2026-02-20T17:58:54.690 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920780 2026-02-20T17:58:54.690 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836495 1-42949672976 2-64424509454 3-317827579907 4-103079215117 5-120259084302 6-141733920780' 2026-02-20T17:58:54.691 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:58:54.691 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-20T17:58:54.755 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757260 2026-02-20T17:58:54.755 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757260 2026-02-20T17:58:54.755 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836495 1-42949672976 2-64424509454 3-317827579907 4-103079215117 5-120259084302 6-141733920780 7-163208757260' 2026-02-20T17:58:54.755 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:58:54.756 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-20T17:58:54.821 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=180388626443 2026-02-20T17:58:54.821 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 180388626443 2026-02-20T17:58:54.822 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836495 1-42949672976 2-64424509454 3-317827579907 4-103079215117 5-120259084302 6-141733920780 7-163208757260 8-180388626443' 2026-02-20T17:58:54.822 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:58:54.822 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-20T17:58:54.885 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=197568495628 2026-02-20T17:58:54.885 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 197568495628 2026-02-20T17:58:54.885 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836495 1-42949672976 2-64424509454 3-317827579907 4-103079215117 5-120259084302 6-141733920780 7-163208757260 8-180388626443 9-197568495628' 2026-02-20T17:58:54.885 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:58:54.885 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.10 flush_pg_stats 2026-02-20T17:58:54.948 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=214748364810 2026-02-20T17:58:54.949 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 214748364810 2026-02-20T17:58:54.949 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836495 1-42949672976 2-64424509454 3-317827579907 4-103079215117 5-120259084302 6-141733920780 7-163208757260 8-180388626443 9-197568495628 10-214748364810' 2026-02-20T17:58:54.949 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:58:54.949 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836495 2026-02-20T17:58:54.949 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:58:54.949 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T17:58:54.950 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836495 2026-02-20T17:58:54.950 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:58:54.952 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836495 2026-02-20T17:58:54.952 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836495' 2026-02-20T17:58:54.952 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.0 seq 21474836495 2026-02-20T17:58:54.952 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T17:58:55.175 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836493 -lt 21474836495 2026-02-20T17:58:55.176 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T17:58:56.176 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T17:58:56.177 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T17:58:56.407 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836496 -lt 21474836495 2026-02-20T17:58:56.408 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:58:56.408 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672976 2026-02-20T17:58:56.408 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:58:56.408 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T17:58:56.409 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672976 2026-02-20T17:58:56.409 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:58:56.410 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672976 2026-02-20T17:58:56.410 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672976' 2026-02-20T17:58:56.410 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.1 seq 42949672976 2026-02-20T17:58:56.411 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T17:58:56.636 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672976 -lt 42949672976 2026-02-20T17:58:56.636 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:58:56.637 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509454 2026-02-20T17:58:56.637 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:58:56.637 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T17:58:56.638 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509454 2026-02-20T17:58:56.638 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:58:56.638 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509454 2026-02-20T17:58:56.639 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509454' 2026-02-20T17:58:56.639 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.2 seq 64424509454 2026-02-20T17:58:56.639 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T17:58:56.865 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509455 -lt 64424509454 2026-02-20T17:58:56.865 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:58:56.865 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-317827579907 2026-02-20T17:58:56.865 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:58:56.866 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T17:58:56.866 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-317827579907 2026-02-20T17:58:56.867 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:58:56.868 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=317827579907 2026-02-20T17:58:56.868 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 317827579907' 2026-02-20T17:58:56.868 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.3 seq 317827579907 2026-02-20T17:58:56.868 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T17:58:57.094 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 317827579907 -lt 317827579907 2026-02-20T17:58:57.094 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:58:57.095 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-103079215117 2026-02-20T17:58:57.095 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:58:57.095 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-20T17:58:57.096 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-103079215117 2026-02-20T17:58:57.096 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:58:57.097 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=103079215117 2026-02-20T17:58:57.097 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 103079215117' 2026-02-20T17:58:57.097 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.4 seq 103079215117 2026-02-20T17:58:57.098 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-20T17:58:57.319 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 103079215118 -lt 103079215117 2026-02-20T17:58:57.320 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:58:57.320 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-120259084302 2026-02-20T17:58:57.320 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:58:57.321 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-20T17:58:57.321 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-120259084302 2026-02-20T17:58:57.321 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:58:57.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=120259084302 2026-02-20T17:58:57.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 120259084302' 2026-02-20T17:58:57.322 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.5 seq 120259084302 2026-02-20T17:58:57.323 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-20T17:58:57.545 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 120259084302 -lt 120259084302 2026-02-20T17:58:57.546 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:58:57.546 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-141733920780 2026-02-20T17:58:57.546 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:58:57.547 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-20T17:58:57.547 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-141733920780 2026-02-20T17:58:57.547 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:58:57.548 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=141733920780 2026-02-20T17:58:57.548 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 141733920780' 2026-02-20T17:58:57.548 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.6 seq 141733920780 2026-02-20T17:58:57.549 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-20T17:58:57.776 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 141733920781 -lt 141733920780 2026-02-20T17:58:57.776 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:58:57.777 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-163208757260 2026-02-20T17:58:57.777 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:58:57.778 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-20T17:58:57.778 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-163208757260 2026-02-20T17:58:57.778 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:58:57.779 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=163208757260 2026-02-20T17:58:57.779 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 163208757260' 2026-02-20T17:58:57.780 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.7 seq 163208757260 2026-02-20T17:58:57.780 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-20T17:58:58.006 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 163208757260 -lt 163208757260 2026-02-20T17:58:58.006 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:58:58.007 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-180388626443 2026-02-20T17:58:58.007 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:58:58.008 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-20T17:58:58.008 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-180388626443 2026-02-20T17:58:58.008 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:58:58.009 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=180388626443 2026-02-20T17:58:58.010 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 180388626443' 2026-02-20T17:58:58.010 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.8 seq 180388626443 2026-02-20T17:58:58.010 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-20T17:58:58.245 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 180388626444 -lt 180388626443 2026-02-20T17:58:58.245 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:58:58.246 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-197568495628 2026-02-20T17:58:58.246 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:58:58.246 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-20T17:58:58.247 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-197568495628 2026-02-20T17:58:58.247 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:58:58.248 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=197568495628 2026-02-20T17:58:58.248 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 197568495628' 2026-02-20T17:58:58.248 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.9 seq 197568495628 2026-02-20T17:58:58.249 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-20T17:58:58.478 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 197568495628 -lt 197568495628 2026-02-20T17:58:58.479 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:58:58.479 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-214748364810 2026-02-20T17:58:58.479 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:58:58.480 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=10 2026-02-20T17:58:58.480 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 10-214748364810 2026-02-20T17:58:58.480 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:58:58.481 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=214748364810 2026-02-20T17:58:58.481 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.10 seq 214748364810' 2026-02-20T17:58:58.482 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.10 seq 214748364810 2026-02-20T17:58:58.482 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-20T17:58:58.707 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 214748364811 -lt 214748364810 2026-02-20T17:58:58.707 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T17:58:58.708 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T17:58:58.708 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T17:58:59.018 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 16 == 0 2026-02-20T17:58:59.018 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T17:58:59.018 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T17:58:59.018 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T17:58:59.018 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T17:58:59.018 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T17:58:59.018 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T17:58:59.018 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T17:58:59.260 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=16 2026-02-20T17:58:59.260 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T17:58:59.260 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T17:58:59.260 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T17:58:59.575 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 = 16 2026-02-20T17:58:59.575 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T17:58:59.575 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T17:58:59.576 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: objectstore_tool td/test-erasure-code 5 SOMETHINGecpool get-bytes 2026-02-20T17:58:59.576 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-code 2026-02-20T17:58:59.576 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: grep --quiet SECONDecpool 2026-02-20T17:58:59.576 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-20T17:58:59.576 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=5 2026-02-20T17:58:59.576 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-20T17:58:59.576 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-code 5 SOMETHINGecpool get-bytes 2026-02-20T17:58:59.576 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-code 2026-02-20T17:58:59.576 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-20T17:58:59.576 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=5 2026-02-20T17:58:59.576 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-20T17:58:59.576 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-code TERM osd.5 2026-02-20T17:58:59.577 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T17:58:59.577 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T17:58:59.577 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T17:58:59.577 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T17:58:59.577 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T17:58:59.682 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T17:58:59.682 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-code 5 SOMETHINGecpool get-bytes 2026-02-20T17:58:59.682 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-code 2026-02-20T17:58:59.682 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-20T17:58:59.682 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=5 2026-02-20T17:58:59.682 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-20T17:58:59.682 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-code/5 2026-02-20T17:58:59.682 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-code/5 SOMETHINGecpool get-bytes 2026-02-20T17:58:59.783 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-code 5 2026-02-20T17:58:59.783 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-code 2026-02-20T17:58:59.783 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-20T17:58:59.783 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=5 2026-02-20T17:58:59.783 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-20T17:58:59.783 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-code/5 2026-02-20T17:58:59.783 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-20T17:58:59.783 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T17:58:59.783 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T17:58:59.783 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T17:58:59.783 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-code/5' 2026-02-20T17:58:59.783 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-code/5/journal' 2026-02-20T17:58:59.783 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-20T17:58:59.783 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-20T17:58:59.783 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-20T17:58:59.783 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-20T17:58:59.783 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:58:59.783 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:58:59.783 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:58:59.783 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:58:59.783 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.58348 2026-02-20T17:58:59.783 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.58348/$cluster-$name.asok' 2026-02-20T17:58:59.784 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' 2026-02-20T17:58:59.784 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-20T17:58:59.784 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-20T17:58:59.784 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-20T17:58:59.784 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T17:58:59.784 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T17:58:59.784 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T17:58:59.784 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T17:58:59.784 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-20T17:58:59.784 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-20T17:58:59.784 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-code/5 2026-02-20T17:58:59.785 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.5 2026-02-20T17:58:59.785 INFO:tasks.workunit.client.0.trial195.stderr:start osd.5 2026-02-20T17:58:59.785 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 5 --fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/5 --osd-journal=td/test-erasure-code/5/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T17:58:59.785 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-code/5/whoami 2026-02-20T17:58:59.786 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 5 = 5 ']' 2026-02-20T17:58:59.786 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-20T17:58:59.786 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-20T17:58:59.786 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-20T17:58:59.801 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:59.800+0000 7fda2252d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:59.802 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:59.802+0000 7fda2252d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:59.804 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:59.803+0000 7fda2252d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:58:59.972 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:58:59.971+0000 7fda2252d900 -1 Falling back to public interface 2026-02-20T17:59:00.013 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 5 2026-02-20T17:59:00.013 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T17:59:00.013 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-20T17:59:00.013 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T17:59:00.013 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T17:59:00.013 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:59:00.013 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T17:59:00.013 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T17:59:00.014 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:59:00.014 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-20T17:59:00.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T17:59:00.353 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:59:00.352+0000 7fda2252d900 -1 osd.5 76 log_to_monitors true 2026-02-20T17:59:01.238 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T17:59:01.238 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:59:01.239 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T17:59:01.239 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T17:59:01.239 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:59:01.239 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-20T17:59:01.461 INFO:tasks.workunit.client.0.trial195.stderr:osd.5 up in weight 1 up_from 79 up_thru 79 down_at 76 last_clean_interval [28,76) [v2:127.0.0.1:6842/140437144,v1:127.0.0.1:6843/140437144] [v2:127.0.0.1:6844/140437144,v1:127.0.0.1:6845/140437144] exists,up 8d3a4486-c70e-46e7-976f-259ea195d131 2026-02-20T17:59:01.461 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T17:59:01.461 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T17:59:01.461 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T17:59:01.461 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-20T17:59:01.461 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T17:59:01.461 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T17:59:01.461 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T17:59:01.461 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T17:59:01.462 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T17:59:01.462 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T17:59:01.462 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T17:59:01.462 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T17:59:01.462 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T17:59:01.539 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T17:59:01.540 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T17:59:01.540 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T17:59:01.540 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T17:59:01.540 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T17:59:01.540 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T17:59:01.765 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T17:59:01.765 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T17:59:01.766 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T17:59:01.766 INFO:tasks.workunit.client.0.trial195.stderr:3 2026-02-20T17:59:01.766 INFO:tasks.workunit.client.0.trial195.stderr:4 2026-02-20T17:59:01.766 INFO:tasks.workunit.client.0.trial195.stderr:5 2026-02-20T17:59:01.766 INFO:tasks.workunit.client.0.trial195.stderr:6 2026-02-20T17:59:01.766 INFO:tasks.workunit.client.0.trial195.stderr:7 2026-02-20T17:59:01.766 INFO:tasks.workunit.client.0.trial195.stderr:8 2026-02-20T17:59:01.766 INFO:tasks.workunit.client.0.trial195.stderr:9 2026-02-20T17:59:01.766 INFO:tasks.workunit.client.0.trial195.stderr:10' 2026-02-20T17:59:01.766 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T17:59:01.767 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:01.767 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T17:59:01.828 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836499 2026-02-20T17:59:01.828 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836499 2026-02-20T17:59:01.828 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499' 2026-02-20T17:59:01.828 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:01.828 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T17:59:01.894 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672979 2026-02-20T17:59:01.894 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672979 2026-02-20T17:59:01.894 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-42949672979' 2026-02-20T17:59:01.894 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:01.894 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T17:59:01.959 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509458 2026-02-20T17:59:01.959 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509458 2026-02-20T17:59:01.959 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-42949672979 2-64424509458' 2026-02-20T17:59:01.959 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:01.959 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T17:59:02.022 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=317827579910 2026-02-20T17:59:02.022 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 317827579910 2026-02-20T17:59:02.022 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-42949672979 2-64424509458 3-317827579910' 2026-02-20T17:59:02.022 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:02.022 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-20T17:59:02.084 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=103079215121 2026-02-20T17:59:02.084 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 103079215121 2026-02-20T17:59:02.084 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-42949672979 2-64424509458 3-317827579910 4-103079215121' 2026-02-20T17:59:02.084 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:02.084 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-20T17:59:02.148 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=339302416387 2026-02-20T17:59:02.148 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 339302416387 2026-02-20T17:59:02.148 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-42949672979 2-64424509458 3-317827579910 4-103079215121 5-339302416387' 2026-02-20T17:59:02.148 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:02.148 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-20T17:59:02.211 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920784 2026-02-20T17:59:02.211 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920784 2026-02-20T17:59:02.211 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-42949672979 2-64424509458 3-317827579910 4-103079215121 5-339302416387 6-141733920784' 2026-02-20T17:59:02.211 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:02.211 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-20T17:59:02.274 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757263 2026-02-20T17:59:02.274 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757263 2026-02-20T17:59:02.274 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-42949672979 2-64424509458 3-317827579910 4-103079215121 5-339302416387 6-141733920784 7-163208757263' 2026-02-20T17:59:02.274 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:02.275 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-20T17:59:02.338 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=180388626447 2026-02-20T17:59:02.339 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 180388626447 2026-02-20T17:59:02.339 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-42949672979 2-64424509458 3-317827579910 4-103079215121 5-339302416387 6-141733920784 7-163208757263 8-180388626447' 2026-02-20T17:59:02.339 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:02.339 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-20T17:59:02.405 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=197568495631 2026-02-20T17:59:02.405 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 197568495631 2026-02-20T17:59:02.405 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-42949672979 2-64424509458 3-317827579910 4-103079215121 5-339302416387 6-141733920784 7-163208757263 8-180388626447 9-197568495631' 2026-02-20T17:59:02.405 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:02.406 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.10 flush_pg_stats 2026-02-20T17:59:02.469 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=214748364814 2026-02-20T17:59:02.469 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 214748364814 2026-02-20T17:59:02.469 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-42949672979 2-64424509458 3-317827579910 4-103079215121 5-339302416387 6-141733920784 7-163208757263 8-180388626447 9-197568495631 10-214748364814' 2026-02-20T17:59:02.469 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:02.469 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836499 2026-02-20T17:59:02.470 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:02.470 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T17:59:02.471 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836499 2026-02-20T17:59:02.471 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:02.471 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836499 2026-02-20T17:59:02.472 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836499' 2026-02-20T17:59:02.472 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.0 seq 21474836499 2026-02-20T17:59:02.472 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T17:59:02.699 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836499 -lt 21474836499 2026-02-20T17:59:02.699 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:02.699 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672979 2026-02-20T17:59:02.699 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:02.700 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T17:59:02.701 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672979 2026-02-20T17:59:02.701 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:02.702 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672979 2026-02-20T17:59:02.702 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672979' 2026-02-20T17:59:02.702 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.1 seq 42949672979 2026-02-20T17:59:02.702 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T17:59:02.932 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672978 -lt 42949672979 2026-02-20T17:59:02.932 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T17:59:03.933 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T17:59:03.934 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T17:59:04.158 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672980 -lt 42949672979 2026-02-20T17:59:04.158 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:04.158 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509458 2026-02-20T17:59:04.158 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:04.159 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T17:59:04.159 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509458 2026-02-20T17:59:04.159 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:04.160 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509458 2026-02-20T17:59:04.160 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509458' 2026-02-20T17:59:04.160 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.2 seq 64424509458 2026-02-20T17:59:04.161 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T17:59:04.385 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509458 -lt 64424509458 2026-02-20T17:59:04.385 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:04.386 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-317827579910 2026-02-20T17:59:04.386 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:04.386 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T17:59:04.387 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-317827579910 2026-02-20T17:59:04.387 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:04.388 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=317827579910 2026-02-20T17:59:04.388 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 317827579910' 2026-02-20T17:59:04.388 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.3 seq 317827579910 2026-02-20T17:59:04.389 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T17:59:04.611 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 317827579911 -lt 317827579910 2026-02-20T17:59:04.612 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:04.612 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-103079215121 2026-02-20T17:59:04.612 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:04.613 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-20T17:59:04.613 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-103079215121 2026-02-20T17:59:04.613 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:04.615 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=103079215121 2026-02-20T17:59:04.615 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 103079215121' 2026-02-20T17:59:04.615 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.4 seq 103079215121 2026-02-20T17:59:04.615 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-20T17:59:04.841 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 103079215121 -lt 103079215121 2026-02-20T17:59:04.841 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:04.842 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-339302416387 2026-02-20T17:59:04.842 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:04.842 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-20T17:59:04.843 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-339302416387 2026-02-20T17:59:04.843 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:04.844 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=339302416387 2026-02-20T17:59:04.844 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 339302416387' 2026-02-20T17:59:04.844 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.5 seq 339302416387 2026-02-20T17:59:04.844 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-20T17:59:05.066 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 339302416387 -lt 339302416387 2026-02-20T17:59:05.067 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:05.067 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-141733920784 2026-02-20T17:59:05.067 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:05.068 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-20T17:59:05.068 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-141733920784 2026-02-20T17:59:05.068 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:05.069 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=141733920784 2026-02-20T17:59:05.069 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 141733920784' 2026-02-20T17:59:05.070 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.6 seq 141733920784 2026-02-20T17:59:05.070 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-20T17:59:05.293 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 141733920784 -lt 141733920784 2026-02-20T17:59:05.293 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:05.293 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-163208757263 2026-02-20T17:59:05.293 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:05.293 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-20T17:59:05.294 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-163208757263 2026-02-20T17:59:05.294 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:05.295 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=163208757263 2026-02-20T17:59:05.295 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 163208757263' 2026-02-20T17:59:05.295 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.7 seq 163208757263 2026-02-20T17:59:05.296 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-20T17:59:05.520 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 163208757264 -lt 163208757263 2026-02-20T17:59:05.520 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:05.521 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-180388626447 2026-02-20T17:59:05.521 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:05.521 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-20T17:59:05.522 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-180388626447 2026-02-20T17:59:05.522 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:05.523 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=180388626447 2026-02-20T17:59:05.523 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 180388626447' 2026-02-20T17:59:05.523 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.8 seq 180388626447 2026-02-20T17:59:05.523 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-20T17:59:05.745 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 180388626447 -lt 180388626447 2026-02-20T17:59:05.745 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:05.746 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-197568495631 2026-02-20T17:59:05.746 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:05.746 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-20T17:59:05.747 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-197568495631 2026-02-20T17:59:05.747 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:05.748 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=197568495631 2026-02-20T17:59:05.748 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 197568495631' 2026-02-20T17:59:05.748 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.9 seq 197568495631 2026-02-20T17:59:05.748 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-20T17:59:05.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 197568495632 -lt 197568495631 2026-02-20T17:59:05.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:05.975 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-214748364814 2026-02-20T17:59:05.975 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:05.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=10 2026-02-20T17:59:05.976 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 10-214748364814 2026-02-20T17:59:05.976 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:05.977 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=214748364814 2026-02-20T17:59:05.977 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.10 seq 214748364814' 2026-02-20T17:59:05.977 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.10 seq 214748364814 2026-02-20T17:59:05.978 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-20T17:59:06.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 214748364815 -lt 214748364814 2026-02-20T17:59:06.205 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T17:59:06.205 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T17:59:06.205 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T17:59:06.518 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 16 == 0 2026-02-20T17:59:06.518 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T17:59:06.519 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T17:59:06.519 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T17:59:06.519 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T17:59:06.519 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T17:59:06.519 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T17:59:06.519 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T17:59:06.739 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=16 2026-02-20T17:59:06.740 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T17:59:06.740 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T17:59:06.740 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T17:59:07.048 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 = 16 2026-02-20T17:59:07.048 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T17:59:07.049 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T17:59:07.049 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:313: TEST_chunk_mapping: ceph osd erasure-code-profile set remap-profile plugin=lrc 'layers=[ [ "cDD", "" ] ]' mapping=_DD 'crush-steps=[ [ "choose", "osd", 0 ] ]' 2026-02-20T17:59:07.343 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:318: TEST_chunk_mapping: ceph osd erasure-code-profile get remap-profile 2026-02-20T17:59:07.566 INFO:tasks.workunit.client.0.trial195.stdout:crush-device-class= 2026-02-20T17:59:07.566 INFO:tasks.workunit.client.0.trial195.stdout:crush-failure-domain=host 2026-02-20T17:59:07.566 INFO:tasks.workunit.client.0.trial195.stdout:crush-num-failure-domains=0 2026-02-20T17:59:07.566 INFO:tasks.workunit.client.0.trial195.stdout:crush-osds-per-failure-domain=0 2026-02-20T17:59:07.567 INFO:tasks.workunit.client.0.trial195.stdout:crush-root=default 2026-02-20T17:59:07.567 INFO:tasks.workunit.client.0.trial195.stdout:crush-steps=[ [ "choose", "osd", 0 ] ] 2026-02-20T17:59:07.567 INFO:tasks.workunit.client.0.trial195.stdout:k=-1 2026-02-20T17:59:07.567 INFO:tasks.workunit.client.0.trial195.stdout:l=-1 2026-02-20T17:59:07.567 INFO:tasks.workunit.client.0.trial195.stdout:layers=[ [ "cDD", "" ] ] 2026-02-20T17:59:07.567 INFO:tasks.workunit.client.0.trial195.stdout:m=-1 2026-02-20T17:59:07.567 INFO:tasks.workunit.client.0.trial195.stdout:mapping=_DD 2026-02-20T17:59:07.567 INFO:tasks.workunit.client.0.trial195.stdout:plugin=lrc 2026-02-20T17:59:07.573 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:319: TEST_chunk_mapping: create_pool remap-pool 12 12 erasure remap-profile 2026-02-20T17:59:07.574 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create remap-pool 12 12 erasure remap-profile 2026-02-20T17:59:07.920 INFO:tasks.workunit.client.0.trial195.stderr:pool 'remap-pool' already exists 2026-02-20T17:59:07.929 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-20T17:59:08.931 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:327: TEST_chunk_mapping: verify_chunk_mapping td/test-erasure-code remap-pool 1 2 2026-02-20T17:59:08.931 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:281: verify_chunk_mapping: local dir=td/test-erasure-code 2026-02-20T17:59:08.931 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:282: verify_chunk_mapping: local poolname=remap-pool 2026-02-20T17:59:08.931 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:283: verify_chunk_mapping: local first=1 2026-02-20T17:59:08.931 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:284: verify_chunk_mapping: local second=2 2026-02-20T17:59:08.932 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:286: verify_chunk_mapping: chunk_size 2026-02-20T17:59:08.932 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:270: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-20T17:59:08.943 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:270: chunk_size: echo 4096 2026-02-20T17:59:08.943 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:286: verify_chunk_mapping: printf '%*s' 4096 FIRSTremap-pool 2026-02-20T17:59:08.944 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:286: verify_chunk_mapping: chunk_size 2026-02-20T17:59:08.944 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:270: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-20T17:59:08.954 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:270: chunk_size: echo 4096 2026-02-20T17:59:08.954 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:286: verify_chunk_mapping: printf '%*s' 4096 SECONDremap-pool 2026-02-20T17:59:08.955 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:286: verify_chunk_mapping: local 'payload= FIRSTremap-pool SECONDremap-pool' 2026-02-20T17:59:08.955 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:287: verify_chunk_mapping: echo -n ' FIRSTremap-pool SECONDremap-pool' 2026-02-20T17:59:08.955 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:289: verify_chunk_mapping: rados --pool remap-pool put SOMETHINGremap-pool td/test-erasure-code/ORIGINAL 2026-02-20T17:59:08.973 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:290: verify_chunk_mapping: rados --pool remap-pool get SOMETHINGremap-pool td/test-erasure-code/COPY 2026-02-20T17:59:08.993 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:291: verify_chunk_mapping: get_osds remap-pool SOMETHINGremap-pool 2026-02-20T17:59:08.993 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=remap-pool 2026-02-20T17:59:08.994 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHINGremap-pool 2026-02-20T17:59:08.994 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map remap-pool SOMETHINGremap-pool 2026-02-20T17:59:08.994 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T17:59:09.225 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=4 2026-02-20T17:59:09.225 INFO:tasks.workunit.client.0.trial195.stderr:6 2026-02-20T17:59:09.225 INFO:tasks.workunit.client.0.trial195.stderr:9' 2026-02-20T17:59:09.226 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 4 6 9 2026-02-20T17:59:09.226 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:291: verify_chunk_mapping: osds=('4' '6' '9') 2026-02-20T17:59:09.226 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:291: verify_chunk_mapping: local -a osds 2026-02-20T17:59:09.226 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i = 0 )) 2026-02-20T17:59:09.226 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i < 3 )) 2026-02-20T17:59:09.226 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:293: verify_chunk_mapping: ceph daemon osd.4 flush_journal 2026-02-20T17:59:09.282 INFO:tasks.workunit.client.0.trial195.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-20T17:59:09.284 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i++ )) 2026-02-20T17:59:09.284 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i < 3 )) 2026-02-20T17:59:09.284 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:293: verify_chunk_mapping: ceph daemon osd.6 flush_journal 2026-02-20T17:59:09.340 INFO:tasks.workunit.client.0.trial195.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-20T17:59:09.342 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i++ )) 2026-02-20T17:59:09.342 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i < 3 )) 2026-02-20T17:59:09.342 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:293: verify_chunk_mapping: ceph daemon osd.9 flush_journal 2026-02-20T17:59:09.398 INFO:tasks.workunit.client.0.trial195.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-20T17:59:09.400 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i++ )) 2026-02-20T17:59:09.400 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i < 3 )) 2026-02-20T17:59:09.400 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-20T17:59:09.400 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:296: verify_chunk_mapping: rm td/test-erasure-code/COPY 2026-02-20T17:59:09.401 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:298: verify_chunk_mapping: get_osds remap-pool SOMETHINGremap-pool 2026-02-20T17:59:09.402 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=remap-pool 2026-02-20T17:59:09.402 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHINGremap-pool 2026-02-20T17:59:09.402 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map remap-pool SOMETHINGremap-pool 2026-02-20T17:59:09.402 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T17:59:09.630 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=4 2026-02-20T17:59:09.630 INFO:tasks.workunit.client.0.trial195.stderr:6 2026-02-20T17:59:09.631 INFO:tasks.workunit.client.0.trial195.stderr:9' 2026-02-20T17:59:09.631 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 4 6 9 2026-02-20T17:59:09.631 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:298: verify_chunk_mapping: osds=('4' '6' '9') 2026-02-20T17:59:09.631 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:298: verify_chunk_mapping: local -a osds 2026-02-20T17:59:09.631 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:299: verify_chunk_mapping: objectstore_tool td/test-erasure-code 6 SOMETHINGremap-pool get-bytes 2026-02-20T17:59:09.631 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:299: verify_chunk_mapping: grep --quiet FIRSTremap-pool 2026-02-20T17:59:09.631 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-code 2026-02-20T17:59:09.631 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-20T17:59:09.631 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=6 2026-02-20T17:59:09.631 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-20T17:59:09.632 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-code 6 SOMETHINGremap-pool get-bytes 2026-02-20T17:59:09.632 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-code 2026-02-20T17:59:09.632 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-20T17:59:09.632 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=6 2026-02-20T17:59:09.632 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-20T17:59:09.632 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-code TERM osd.6 2026-02-20T17:59:09.632 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T17:59:09.632 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T17:59:09.632 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T17:59:09.632 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T17:59:09.632 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T17:59:09.936 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T17:59:09.937 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-code 6 SOMETHINGremap-pool get-bytes 2026-02-20T17:59:09.937 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-code 2026-02-20T17:59:09.937 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-20T17:59:09.937 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=6 2026-02-20T17:59:09.937 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-20T17:59:09.937 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-code/6 2026-02-20T17:59:09.937 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-code/6 SOMETHINGremap-pool get-bytes 2026-02-20T17:59:10.035 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-code 6 2026-02-20T17:59:10.035 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-code 2026-02-20T17:59:10.035 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-20T17:59:10.035 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=6 2026-02-20T17:59:10.035 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-20T17:59:10.035 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-code/6 2026-02-20T17:59:10.035 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-20T17:59:10.035 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T17:59:10.035 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T17:59:10.035 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T17:59:10.035 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-code/6' 2026-02-20T17:59:10.035 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-code/6/journal' 2026-02-20T17:59:10.035 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-20T17:59:10.035 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-20T17:59:10.035 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-20T17:59:10.035 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-20T17:59:10.035 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:59:10.035 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:59:10.035 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:59:10.035 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:59:10.035 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.58348 2026-02-20T17:59:10.035 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.58348/$cluster-$name.asok' 2026-02-20T17:59:10.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' 2026-02-20T17:59:10.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-20T17:59:10.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-20T17:59:10.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-20T17:59:10.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T17:59:10.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T17:59:10.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T17:59:10.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T17:59:10.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-20T17:59:10.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-20T17:59:10.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-code/6 2026-02-20T17:59:10.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.6 2026-02-20T17:59:10.036 INFO:tasks.workunit.client.0.trial195.stderr:start osd.6 2026-02-20T17:59:10.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 6 --fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/6 --osd-journal=td/test-erasure-code/6/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T17:59:10.036 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-code/6/whoami 2026-02-20T17:59:10.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 6 = 6 ']' 2026-02-20T17:59:10.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-20T17:59:10.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-20T17:59:10.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-20T17:59:10.050 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:59:10.049+0000 7f3188e63900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:59:10.051 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:59:10.050+0000 7f3188e63900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:59:10.052 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:59:10.051+0000 7f3188e63900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:59:10.219 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:59:10.217+0000 7f3188e63900 -1 Falling back to public interface 2026-02-20T17:59:10.260 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 6 2026-02-20T17:59:10.260 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T17:59:10.260 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-20T17:59:10.260 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T17:59:10.261 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T17:59:10.261 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:59:10.261 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T17:59:10.261 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T17:59:10.261 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:59:10.261 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-20T17:59:10.486 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T17:59:10.584 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:59:10.583+0000 7f3188e63900 -1 osd.6 86 log_to_monitors true 2026-02-20T17:59:11.488 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T17:59:11.488 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:59:11.488 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T17:59:11.488 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T17:59:11.489 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:59:11.489 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-20T17:59:11.714 INFO:tasks.workunit.client.0.trial195.stderr:osd.6 up in weight 1 up_from 90 up_thru 90 down_at 87 last_clean_interval [33,86) [v2:127.0.0.1:6850/1973528319,v1:127.0.0.1:6851/1973528319] [v2:127.0.0.1:6852/1973528319,v1:127.0.0.1:6853/1973528319] exists,up fed68fe0-32a7-4c8f-b0e1-51e9d8edb65f 2026-02-20T17:59:11.714 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T17:59:11.714 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T17:59:11.715 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T17:59:11.715 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-20T17:59:11.715 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T17:59:11.715 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T17:59:11.715 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T17:59:11.715 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T17:59:11.715 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T17:59:11.715 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T17:59:11.715 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T17:59:11.715 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T17:59:11.715 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T17:59:11.794 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T17:59:11.794 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T17:59:11.794 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T17:59:11.794 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T17:59:11.794 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T17:59:11.795 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T17:59:12.021 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T17:59:12.022 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T17:59:12.022 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T17:59:12.022 INFO:tasks.workunit.client.0.trial195.stderr:3 2026-02-20T17:59:12.022 INFO:tasks.workunit.client.0.trial195.stderr:4 2026-02-20T17:59:12.022 INFO:tasks.workunit.client.0.trial195.stderr:5 2026-02-20T17:59:12.022 INFO:tasks.workunit.client.0.trial195.stderr:6 2026-02-20T17:59:12.022 INFO:tasks.workunit.client.0.trial195.stderr:7 2026-02-20T17:59:12.022 INFO:tasks.workunit.client.0.trial195.stderr:8 2026-02-20T17:59:12.022 INFO:tasks.workunit.client.0.trial195.stderr:9 2026-02-20T17:59:12.023 INFO:tasks.workunit.client.0.trial195.stderr:10' 2026-02-20T17:59:12.023 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T17:59:12.023 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:12.023 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T17:59:12.086 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836503 2026-02-20T17:59:12.086 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836503 2026-02-20T17:59:12.086 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836503' 2026-02-20T17:59:12.087 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:12.087 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T17:59:12.150 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672983 2026-02-20T17:59:12.150 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672983 2026-02-20T17:59:12.150 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836503 1-42949672983' 2026-02-20T17:59:12.151 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:12.151 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T17:59:12.216 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509462 2026-02-20T17:59:12.216 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509462 2026-02-20T17:59:12.217 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836503 1-42949672983 2-64424509462' 2026-02-20T17:59:12.217 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:12.217 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T17:59:12.279 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=317827579914 2026-02-20T17:59:12.279 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 317827579914 2026-02-20T17:59:12.279 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836503 1-42949672983 2-64424509462 3-317827579914' 2026-02-20T17:59:12.279 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:12.280 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-20T17:59:12.343 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=103079215125 2026-02-20T17:59:12.344 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 103079215125 2026-02-20T17:59:12.344 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836503 1-42949672983 2-64424509462 3-317827579914 4-103079215125' 2026-02-20T17:59:12.344 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:12.344 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-20T17:59:12.407 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=339302416391 2026-02-20T17:59:12.407 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 339302416391 2026-02-20T17:59:12.407 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836503 1-42949672983 2-64424509462 3-317827579914 4-103079215125 5-339302416391' 2026-02-20T17:59:12.407 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:12.408 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-20T17:59:12.468 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=386547056643 2026-02-20T17:59:12.469 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 386547056643 2026-02-20T17:59:12.469 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836503 1-42949672983 2-64424509462 3-317827579914 4-103079215125 5-339302416391 6-386547056643' 2026-02-20T17:59:12.469 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:12.469 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-20T17:59:12.535 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757267 2026-02-20T17:59:12.535 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757267 2026-02-20T17:59:12.535 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836503 1-42949672983 2-64424509462 3-317827579914 4-103079215125 5-339302416391 6-386547056643 7-163208757267' 2026-02-20T17:59:12.536 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:12.536 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-20T17:59:12.600 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=180388626451 2026-02-20T17:59:12.600 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 180388626451 2026-02-20T17:59:12.600 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836503 1-42949672983 2-64424509462 3-317827579914 4-103079215125 5-339302416391 6-386547056643 7-163208757267 8-180388626451' 2026-02-20T17:59:12.600 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:12.600 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-20T17:59:12.666 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=197568495635 2026-02-20T17:59:12.666 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 197568495635 2026-02-20T17:59:12.666 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836503 1-42949672983 2-64424509462 3-317827579914 4-103079215125 5-339302416391 6-386547056643 7-163208757267 8-180388626451 9-197568495635' 2026-02-20T17:59:12.667 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:12.667 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.10 flush_pg_stats 2026-02-20T17:59:12.728 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=214748364818 2026-02-20T17:59:12.729 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 214748364818 2026-02-20T17:59:12.729 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836503 1-42949672983 2-64424509462 3-317827579914 4-103079215125 5-339302416391 6-386547056643 7-163208757267 8-180388626451 9-197568495635 10-214748364818' 2026-02-20T17:59:12.729 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:12.729 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836503 2026-02-20T17:59:12.729 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:12.730 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T17:59:12.730 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836503 2026-02-20T17:59:12.730 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:12.731 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836503 2026-02-20T17:59:12.731 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836503' 2026-02-20T17:59:12.731 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.0 seq 21474836503 2026-02-20T17:59:12.731 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T17:59:12.961 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836501 -lt 21474836503 2026-02-20T17:59:12.961 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T17:59:13.962 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T17:59:13.962 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T17:59:14.187 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836503 -lt 21474836503 2026-02-20T17:59:14.187 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:14.187 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672983 2026-02-20T17:59:14.188 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:14.188 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T17:59:14.189 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672983 2026-02-20T17:59:14.189 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:14.190 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672983 2026-02-20T17:59:14.190 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672983' 2026-02-20T17:59:14.190 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.1 seq 42949672983 2026-02-20T17:59:14.190 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T17:59:14.411 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672984 -lt 42949672983 2026-02-20T17:59:14.411 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:14.411 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509462 2026-02-20T17:59:14.411 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:14.412 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T17:59:14.413 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509462 2026-02-20T17:59:14.413 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:14.414 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509462 2026-02-20T17:59:14.414 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509462' 2026-02-20T17:59:14.414 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.2 seq 64424509462 2026-02-20T17:59:14.414 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T17:59:14.635 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509462 -lt 64424509462 2026-02-20T17:59:14.636 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:14.636 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-317827579914 2026-02-20T17:59:14.636 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:14.636 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T17:59:14.637 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-317827579914 2026-02-20T17:59:14.637 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:14.638 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=317827579914 2026-02-20T17:59:14.638 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 317827579914' 2026-02-20T17:59:14.638 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.3 seq 317827579914 2026-02-20T17:59:14.638 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T17:59:14.862 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 317827579915 -lt 317827579914 2026-02-20T17:59:14.862 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:14.862 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-103079215125 2026-02-20T17:59:14.863 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:14.863 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-20T17:59:14.863 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-103079215125 2026-02-20T17:59:14.864 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:14.865 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=103079215125 2026-02-20T17:59:14.865 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 103079215125' 2026-02-20T17:59:14.865 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.4 seq 103079215125 2026-02-20T17:59:14.865 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-20T17:59:15.094 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 103079215125 -lt 103079215125 2026-02-20T17:59:15.094 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:15.095 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-339302416391 2026-02-20T17:59:15.095 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:15.096 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-20T17:59:15.096 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-339302416391 2026-02-20T17:59:15.096 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:15.098 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=339302416391 2026-02-20T17:59:15.098 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 339302416391' 2026-02-20T17:59:15.098 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.5 seq 339302416391 2026-02-20T17:59:15.098 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-20T17:59:15.328 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 339302416391 -lt 339302416391 2026-02-20T17:59:15.328 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:15.328 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-386547056643 2026-02-20T17:59:15.328 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:15.329 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-20T17:59:15.329 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-386547056643 2026-02-20T17:59:15.330 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:15.330 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=386547056643 2026-02-20T17:59:15.331 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 386547056643' 2026-02-20T17:59:15.331 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.6 seq 386547056643 2026-02-20T17:59:15.331 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-20T17:59:15.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 386547056643 -lt 386547056643 2026-02-20T17:59:15.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:15.555 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-163208757267 2026-02-20T17:59:15.555 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:15.555 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-20T17:59:15.556 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-163208757267 2026-02-20T17:59:15.556 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:15.557 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=163208757267 2026-02-20T17:59:15.557 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 163208757267' 2026-02-20T17:59:15.557 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.7 seq 163208757267 2026-02-20T17:59:15.557 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-20T17:59:15.784 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 163208757268 -lt 163208757267 2026-02-20T17:59:15.784 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:15.784 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-180388626451 2026-02-20T17:59:15.784 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:15.785 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-20T17:59:15.786 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-180388626451 2026-02-20T17:59:15.786 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:15.787 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=180388626451 2026-02-20T17:59:15.787 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 180388626451' 2026-02-20T17:59:15.787 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.8 seq 180388626451 2026-02-20T17:59:15.787 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-20T17:59:16.011 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 180388626452 -lt 180388626451 2026-02-20T17:59:16.011 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:16.011 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-197568495635 2026-02-20T17:59:16.012 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:16.012 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-20T17:59:16.013 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-197568495635 2026-02-20T17:59:16.013 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:16.014 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=197568495635 2026-02-20T17:59:16.014 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 197568495635' 2026-02-20T17:59:16.014 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.9 seq 197568495635 2026-02-20T17:59:16.014 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-20T17:59:16.239 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 197568495636 -lt 197568495635 2026-02-20T17:59:16.240 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:16.240 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-214748364818 2026-02-20T17:59:16.240 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:16.241 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=10 2026-02-20T17:59:16.241 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 10-214748364818 2026-02-20T17:59:16.241 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:16.242 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=214748364818 2026-02-20T17:59:16.242 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.10 seq 214748364818' 2026-02-20T17:59:16.242 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.10 seq 214748364818 2026-02-20T17:59:16.243 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-20T17:59:16.472 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 214748364819 -lt 214748364818 2026-02-20T17:59:16.473 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T17:59:16.473 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T17:59:16.473 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T17:59:16.783 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-20T17:59:16.783 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T17:59:16.784 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T17:59:16.784 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T17:59:16.784 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T17:59:16.784 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T17:59:16.784 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T17:59:16.784 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T17:59:17.003 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-20T17:59:17.003 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T17:59:17.004 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T17:59:17.004 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T17:59:17.318 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-20T17:59:17.318 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T17:59:17.318 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T17:59:17.320 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: objectstore_tool td/test-erasure-code 9 SOMETHINGremap-pool get-bytes 2026-02-20T17:59:17.320 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-code 2026-02-20T17:59:17.320 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: grep --quiet SECONDremap-pool 2026-02-20T17:59:17.320 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-20T17:59:17.320 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=9 2026-02-20T17:59:17.320 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-20T17:59:17.320 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-code 9 SOMETHINGremap-pool get-bytes 2026-02-20T17:59:17.321 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-code 2026-02-20T17:59:17.321 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-20T17:59:17.321 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=9 2026-02-20T17:59:17.321 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-20T17:59:17.321 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-code TERM osd.9 2026-02-20T17:59:17.321 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T17:59:17.321 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T17:59:17.321 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T17:59:17.321 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T17:59:17.321 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T17:59:17.425 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T17:59:17.425 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-code 9 SOMETHINGremap-pool get-bytes 2026-02-20T17:59:17.425 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-code 2026-02-20T17:59:17.425 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-20T17:59:17.425 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=9 2026-02-20T17:59:17.425 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-20T17:59:17.425 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-code/9 2026-02-20T17:59:17.425 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-code/9 SOMETHINGremap-pool get-bytes 2026-02-20T17:59:17.523 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-code 9 2026-02-20T17:59:17.524 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-code 2026-02-20T17:59:17.524 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-20T17:59:17.524 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=9 2026-02-20T17:59:17.524 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-20T17:59:17.524 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-code/9 2026-02-20T17:59:17.524 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-20T17:59:17.524 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T17:59:17.524 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T17:59:17.524 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T17:59:17.524 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-code/9' 2026-02-20T17:59:17.524 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-code/9/journal' 2026-02-20T17:59:17.525 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-20T17:59:17.525 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-20T17:59:17.525 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-20T17:59:17.525 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-20T17:59:17.525 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T17:59:17.525 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T17:59:17.525 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T17:59:17.525 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T17:59:17.525 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.58348 2026-02-20T17:59:17.525 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.58348/$cluster-$name.asok' 2026-02-20T17:59:17.525 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' 2026-02-20T17:59:17.525 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-20T17:59:17.526 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-20T17:59:17.526 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-20T17:59:17.526 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T17:59:17.526 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T17:59:17.526 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T17:59:17.526 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T17:59:17.526 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-20T17:59:17.526 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-20T17:59:17.526 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-code/9 2026-02-20T17:59:17.526 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.9 2026-02-20T17:59:17.526 INFO:tasks.workunit.client.0.trial195.stderr:start osd.9 2026-02-20T17:59:17.526 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 9 --fsid=7050e8a5-40a5-4016-a700-d87e8ecfba2a --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/9 --osd-journal=td/test-erasure-code/9/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.58348/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T17:59:17.527 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-code/9/whoami 2026-02-20T17:59:17.527 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 9 = 9 ']' 2026-02-20T17:59:17.527 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-20T17:59:17.528 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-20T17:59:17.528 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-20T17:59:17.539 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:59:17.538+0000 7f20e3527900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:59:17.540 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:59:17.539+0000 7f20e3527900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:59:17.541 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:59:17.540+0000 7f20e3527900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T17:59:17.683 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:59:17.682+0000 7f20e3527900 -1 Falling back to public interface 2026-02-20T17:59:17.750 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 9 2026-02-20T17:59:17.750 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T17:59:17.751 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=9 2026-02-20T17:59:17.751 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T17:59:17.751 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T17:59:17.751 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:59:17.751 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T17:59:17.751 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T17:59:17.751 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:59:17.751 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-20T17:59:17.977 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T17:59:18.025 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T17:59:18.024+0000 7f20e3527900 -1 osd.9 92 log_to_monitors true 2026-02-20T17:59:18.979 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T17:59:18.979 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T17:59:18.979 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T17:59:18.979 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T17:59:18.979 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T17:59:18.979 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-20T17:59:19.203 INFO:tasks.workunit.client.0.trial195.stderr:osd.9 up in weight 1 up_from 95 up_thru 95 down_at 92 last_clean_interval [46,92) [v2:127.0.0.1:6874/1058046658,v1:127.0.0.1:6875/1058046658] [v2:127.0.0.1:6876/1058046658,v1:127.0.0.1:6877/1058046658] exists,up c0629070-187c-4594-aae4-4fd19faed37e 2026-02-20T17:59:19.203 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T17:59:19.203 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T17:59:19.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T17:59:19.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-20T17:59:19.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T17:59:19.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T17:59:19.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T17:59:19.204 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T17:59:19.204 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T17:59:19.204 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T17:59:19.204 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T17:59:19.204 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T17:59:19.205 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T17:59:19.281 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T17:59:19.281 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T17:59:19.282 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T17:59:19.282 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T17:59:19.282 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T17:59:19.282 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T17:59:19.513 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T17:59:19.513 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T17:59:19.513 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T17:59:19.513 INFO:tasks.workunit.client.0.trial195.stderr:3 2026-02-20T17:59:19.513 INFO:tasks.workunit.client.0.trial195.stderr:4 2026-02-20T17:59:19.513 INFO:tasks.workunit.client.0.trial195.stderr:5 2026-02-20T17:59:19.514 INFO:tasks.workunit.client.0.trial195.stderr:6 2026-02-20T17:59:19.514 INFO:tasks.workunit.client.0.trial195.stderr:7 2026-02-20T17:59:19.514 INFO:tasks.workunit.client.0.trial195.stderr:8 2026-02-20T17:59:19.514 INFO:tasks.workunit.client.0.trial195.stderr:9 2026-02-20T17:59:19.514 INFO:tasks.workunit.client.0.trial195.stderr:10' 2026-02-20T17:59:19.514 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T17:59:19.514 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:19.514 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T17:59:19.577 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836506 2026-02-20T17:59:19.578 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836506 2026-02-20T17:59:19.578 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836506' 2026-02-20T17:59:19.578 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:19.578 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T17:59:19.640 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672987 2026-02-20T17:59:19.641 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672987 2026-02-20T17:59:19.641 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836506 1-42949672987' 2026-02-20T17:59:19.641 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:19.641 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T17:59:19.704 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509465 2026-02-20T17:59:19.705 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509465 2026-02-20T17:59:19.705 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836506 1-42949672987 2-64424509465' 2026-02-20T17:59:19.705 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:19.705 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T17:59:19.765 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=317827579918 2026-02-20T17:59:19.765 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 317827579918 2026-02-20T17:59:19.765 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836506 1-42949672987 2-64424509465 3-317827579918' 2026-02-20T17:59:19.765 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:19.765 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-20T17:59:19.828 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=103079215128 2026-02-20T17:59:19.829 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 103079215128 2026-02-20T17:59:19.829 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836506 1-42949672987 2-64424509465 3-317827579918 4-103079215128' 2026-02-20T17:59:19.829 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:19.829 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-20T17:59:19.893 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=339302416394 2026-02-20T17:59:19.894 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 339302416394 2026-02-20T17:59:19.894 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836506 1-42949672987 2-64424509465 3-317827579918 4-103079215128 5-339302416394' 2026-02-20T17:59:19.894 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:19.894 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-20T17:59:19.958 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=386547056646 2026-02-20T17:59:19.958 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 386547056646 2026-02-20T17:59:19.958 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836506 1-42949672987 2-64424509465 3-317827579918 4-103079215128 5-339302416394 6-386547056646' 2026-02-20T17:59:19.958 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:19.958 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-20T17:59:20.021 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757271 2026-02-20T17:59:20.021 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757271 2026-02-20T17:59:20.021 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836506 1-42949672987 2-64424509465 3-317827579918 4-103079215128 5-339302416394 6-386547056646 7-163208757271' 2026-02-20T17:59:20.021 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:20.021 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-20T17:59:20.085 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=180388626454 2026-02-20T17:59:20.085 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 180388626454 2026-02-20T17:59:20.085 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836506 1-42949672987 2-64424509465 3-317827579918 4-103079215128 5-339302416394 6-386547056646 7-163208757271 8-180388626454' 2026-02-20T17:59:20.085 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:20.086 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-20T17:59:20.149 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=408021893123 2026-02-20T17:59:20.149 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 408021893123 2026-02-20T17:59:20.149 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836506 1-42949672987 2-64424509465 3-317827579918 4-103079215128 5-339302416394 6-386547056646 7-163208757271 8-180388626454 9-408021893123' 2026-02-20T17:59:20.149 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:20.150 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.10 flush_pg_stats 2026-02-20T17:59:20.215 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=214748364821 2026-02-20T17:59:20.215 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 214748364821 2026-02-20T17:59:20.215 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836506 1-42949672987 2-64424509465 3-317827579918 4-103079215128 5-339302416394 6-386547056646 7-163208757271 8-180388626454 9-408021893123 10-214748364821' 2026-02-20T17:59:20.215 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:20.215 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836506 2026-02-20T17:59:20.215 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:20.216 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T17:59:20.216 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836506 2026-02-20T17:59:20.216 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:20.217 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836506 2026-02-20T17:59:20.217 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836506' 2026-02-20T17:59:20.218 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.0 seq 21474836506 2026-02-20T17:59:20.218 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T17:59:20.442 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836506 -lt 21474836506 2026-02-20T17:59:20.442 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:20.443 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672987 2026-02-20T17:59:20.443 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:20.444 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T17:59:20.444 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672987 2026-02-20T17:59:20.444 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:20.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672987 2026-02-20T17:59:20.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672987' 2026-02-20T17:59:20.446 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.1 seq 42949672987 2026-02-20T17:59:20.446 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T17:59:20.672 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672987 -lt 42949672987 2026-02-20T17:59:20.672 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:20.673 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509465 2026-02-20T17:59:20.673 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:20.673 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T17:59:20.673 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509465 2026-02-20T17:59:20.674 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:20.674 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509465 2026-02-20T17:59:20.675 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509465' 2026-02-20T17:59:20.675 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.2 seq 64424509465 2026-02-20T17:59:20.675 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T17:59:20.899 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509465 -lt 64424509465 2026-02-20T17:59:20.900 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:20.900 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-317827579918 2026-02-20T17:59:20.900 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:20.901 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T17:59:20.901 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-317827579918 2026-02-20T17:59:20.901 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:20.903 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=317827579918 2026-02-20T17:59:20.903 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 317827579918' 2026-02-20T17:59:20.903 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.3 seq 317827579918 2026-02-20T17:59:20.903 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T17:59:21.131 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 317827579918 -lt 317827579918 2026-02-20T17:59:21.131 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:21.131 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-103079215128 2026-02-20T17:59:21.131 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:21.132 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-20T17:59:21.132 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-103079215128 2026-02-20T17:59:21.133 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:21.133 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=103079215128 2026-02-20T17:59:21.134 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 103079215128' 2026-02-20T17:59:21.134 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.4 seq 103079215128 2026-02-20T17:59:21.134 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-20T17:59:21.357 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 103079215128 -lt 103079215128 2026-02-20T17:59:21.357 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:21.357 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-339302416394 2026-02-20T17:59:21.357 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:21.358 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-20T17:59:21.358 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-339302416394 2026-02-20T17:59:21.358 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:21.360 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=339302416394 2026-02-20T17:59:21.360 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 339302416394' 2026-02-20T17:59:21.360 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.5 seq 339302416394 2026-02-20T17:59:21.360 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-20T17:59:21.586 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 339302416393 -lt 339302416394 2026-02-20T17:59:21.587 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T17:59:22.587 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T17:59:22.588 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-20T17:59:22.814 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 339302416395 -lt 339302416394 2026-02-20T17:59:22.814 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:22.814 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-386547056646 2026-02-20T17:59:22.815 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:22.815 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-20T17:59:22.816 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-386547056646 2026-02-20T17:59:22.816 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:22.817 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=386547056646 2026-02-20T17:59:22.817 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 386547056646' 2026-02-20T17:59:22.817 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.6 seq 386547056646 2026-02-20T17:59:22.817 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-20T17:59:23.042 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 386547056647 -lt 386547056646 2026-02-20T17:59:23.043 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:23.043 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-163208757271 2026-02-20T17:59:23.043 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:23.044 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-20T17:59:23.045 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-163208757271 2026-02-20T17:59:23.045 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:23.046 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=163208757271 2026-02-20T17:59:23.046 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 163208757271' 2026-02-20T17:59:23.046 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.7 seq 163208757271 2026-02-20T17:59:23.046 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-20T17:59:23.268 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 163208757271 -lt 163208757271 2026-02-20T17:59:23.268 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:23.269 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-180388626454 2026-02-20T17:59:23.269 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:23.269 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-20T17:59:23.270 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-180388626454 2026-02-20T17:59:23.270 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:23.271 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=180388626454 2026-02-20T17:59:23.271 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 180388626454' 2026-02-20T17:59:23.271 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.8 seq 180388626454 2026-02-20T17:59:23.271 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-20T17:59:23.498 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 180388626455 -lt 180388626454 2026-02-20T17:59:23.498 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:23.499 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-408021893123 2026-02-20T17:59:23.499 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:23.500 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-20T17:59:23.500 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-408021893123 2026-02-20T17:59:23.500 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:23.501 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=408021893123 2026-02-20T17:59:23.501 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 408021893123' 2026-02-20T17:59:23.501 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.9 seq 408021893123 2026-02-20T17:59:23.501 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-20T17:59:23.727 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 408021893123 -lt 408021893123 2026-02-20T17:59:23.727 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:23.728 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-214748364821 2026-02-20T17:59:23.728 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:23.729 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=10 2026-02-20T17:59:23.729 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 10-214748364821 2026-02-20T17:59:23.729 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:23.730 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=214748364821 2026-02-20T17:59:23.730 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.10 seq 214748364821' 2026-02-20T17:59:23.731 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.10 seq 214748364821 2026-02-20T17:59:23.731 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-20T17:59:23.955 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 214748364822 -lt 214748364821 2026-02-20T17:59:23.956 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T17:59:23.956 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T17:59:23.956 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T17:59:24.268 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-20T17:59:24.268 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T17:59:24.269 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T17:59:24.269 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T17:59:24.269 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T17:59:24.269 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T17:59:24.269 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T17:59:24.269 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T17:59:24.490 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-20T17:59:24.491 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T17:59:24.491 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T17:59:24.491 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T17:59:24.807 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-20T17:59:24.807 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T17:59:24.807 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T17:59:24.808 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:329: TEST_chunk_mapping: delete_pool remap-pool 2026-02-20T17:59:24.808 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=remap-pool 2026-02-20T17:59:24.808 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete remap-pool remap-pool --yes-i-really-really-mean-it 2026-02-20T17:59:25.109 INFO:tasks.workunit.client.0.trial195.stderr:pool 'remap-pool' does not exist 2026-02-20T17:59:25.117 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:330: TEST_chunk_mapping: ceph osd erasure-code-profile rm remap-profile 2026-02-20T17:59:25.394 INFO:tasks.workunit.client.0.trial195.stderr:erasure-code-profile remap-profile does not exist 2026-02-20T17:59:25.402 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-20T17:59:25.402 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_isa td/test-erasure-code 2026-02-20T17:59:25.403 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:193: TEST_rados_put_get_isa: erasure_code_plugin_exists isa 2026-02-20T17:59:25.403 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2104: erasure_code_plugin_exists: local plugin=isa 2026-02-20T17:59:25.403 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2105: erasure_code_plugin_exists: local status 2026-02-20T17:59:25.403 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2106: erasure_code_plugin_exists: local grepstr 2026-02-20T17:59:25.403 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2107: erasure_code_plugin_exists: local s 2026-02-20T17:59:25.403 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2108: erasure_code_plugin_exists: case `uname` in 2026-02-20T17:59:25.403 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2108: erasure_code_plugin_exists: uname 2026-02-20T17:59:25.404 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2110: erasure_code_plugin_exists: grepstr='isa.*No such file' 2026-02-20T17:59:25.404 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2113: erasure_code_plugin_exists: ceph osd erasure-code-profile set TESTPROFILE plugin=isa 2026-02-20T17:59:25.692 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2113: erasure_code_plugin_exists: s= 2026-02-20T17:59:25.692 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2114: erasure_code_plugin_exists: local status=0 2026-02-20T17:59:25.692 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2115: erasure_code_plugin_exists: '[' 0 -eq 0 ']' 2026-02-20T17:59:25.693 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2116: erasure_code_plugin_exists: ceph osd erasure-code-profile rm TESTPROFILE 2026-02-20T17:59:25.966 INFO:tasks.workunit.client.0.trial195.stderr:erasure-code-profile TESTPROFILE does not exist 2026-02-20T17:59:25.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2125: erasure_code_plugin_exists: return 0 2026-02-20T17:59:25.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:197: TEST_rados_put_get_isa: local dir=td/test-erasure-code 2026-02-20T17:59:25.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:198: TEST_rados_put_get_isa: local poolname=pool-isa 2026-02-20T17:59:25.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:200: TEST_rados_put_get_isa: ceph osd erasure-code-profile set profile-isa plugin=isa crush-failure-domain=osd 2026-02-20T17:59:26.259 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:203: TEST_rados_put_get_isa: create_pool pool-isa 1 1 erasure profile-isa 2026-02-20T17:59:26.259 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-isa 1 1 erasure profile-isa 2026-02-20T17:59:26.629 INFO:tasks.workunit.client.0.trial195.stderr:pool 'pool-isa' already exists 2026-02-20T17:59:26.637 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-20T17:59:27.639 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:206: TEST_rados_put_get_isa: rados_put_get td/test-erasure-code pool-isa 2026-02-20T17:59:27.640 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-20T17:59:27.640 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-isa 2026-02-20T17:59:27.640 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-20T17:59:27.640 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-20T17:59:27.640 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-20T17:59:27.640 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-20T17:59:27.640 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-20T17:59:27.640 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-20T17:59:27.640 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-20T17:59:27.641 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-20T17:59:27.641 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-20T17:59:27.641 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-isa put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-20T17:59:27.700 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-isa get SOMETHING td/test-erasure-code/COPY 2026-02-20T17:59:27.719 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-20T17:59:27.720 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-20T17:59:27.722 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-isa SOMETHING 2026-02-20T17:59:27.722 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-isa 2026-02-20T17:59:27.722 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-20T17:59:27.722 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-isa SOMETHING 2026-02-20T17:59:27.722 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T17:59:27.952 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=4 2026-02-20T17:59:27.952 INFO:tasks.workunit.client.0.trial195.stderr:10 2026-02-20T17:59:27.952 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T17:59:27.952 INFO:tasks.workunit.client.0.trial195.stderr:7' 2026-02-20T17:59:27.952 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 4 10 0 7 2026-02-20T17:59:27.952 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('4' '10' '0' '7') 2026-02-20T17:59:27.952 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-20T17:59:27.952 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=3 2026-02-20T17:59:27.953 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 7 2026-02-20T17:59:28.228 INFO:tasks.workunit.client.0.trial195.stderr:osd.7 is already out. 2026-02-20T17:59:28.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-20T17:59:33.240 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-isa SOMETHING 2026-02-20T17:59:33.240 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-isa 2026-02-20T17:59:33.240 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-20T17:59:33.240 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<7\>' 2026-02-20T17:59:33.240 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-isa SOMETHING 2026-02-20T17:59:33.240 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T17:59:33.472 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=4 2026-02-20T17:59:33.472 INFO:tasks.workunit.client.0.trial195.stderr:10 2026-02-20T17:59:33.473 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T17:59:33.473 INFO:tasks.workunit.client.0.trial195.stderr:1' 2026-02-20T17:59:33.473 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 4 10 0 1 2026-02-20T17:59:33.473 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-isa get SOMETHING td/test-erasure-code/COPY 2026-02-20T17:59:33.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-20T17:59:33.493 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 7 2026-02-20T17:59:33.773 INFO:tasks.workunit.client.0.trial195.stderr:osd.7 is already in. 2026-02-20T17:59:33.781 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-20T17:59:33.782 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:208: TEST_rados_put_get_isa: delete_pool pool-isa 2026-02-20T17:59:33.783 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-isa 2026-02-20T17:59:33.783 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-isa pool-isa --yes-i-really-really-mean-it 2026-02-20T17:59:34.074 INFO:tasks.workunit.client.0.trial195.stderr:pool 'pool-isa' does not exist 2026-02-20T17:59:34.083 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-20T17:59:34.083 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_jerasure td/test-erasure-code 2026-02-20T17:59:34.083 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:212: TEST_rados_put_get_jerasure: local dir=td/test-erasure-code 2026-02-20T17:59:34.084 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:214: TEST_rados_put_get_jerasure: rados_put_get td/test-erasure-code ecpool 2026-02-20T17:59:34.084 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-20T17:59:34.084 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=ecpool 2026-02-20T17:59:34.084 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-20T17:59:34.084 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-20T17:59:34.084 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-20T17:59:34.084 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-20T17:59:34.084 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-20T17:59:34.084 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-20T17:59:34.084 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-20T17:59:34.085 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-20T17:59:34.085 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-20T17:59:34.085 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool ecpool put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-20T17:59:34.104 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool ecpool get SOMETHING td/test-erasure-code/COPY 2026-02-20T17:59:34.123 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-20T17:59:34.124 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-20T17:59:34.125 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds ecpool SOMETHING 2026-02-20T17:59:34.125 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-20T17:59:34.125 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-20T17:59:34.126 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool SOMETHING 2026-02-20T17:59:34.126 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T17:59:34.354 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=6 2026-02-20T17:59:34.354 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T17:59:34.354 INFO:tasks.workunit.client.0.trial195.stderr:7 2026-02-20T17:59:34.354 INFO:tasks.workunit.client.0.trial195.stderr:9' 2026-02-20T17:59:34.355 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 6 0 7 9 2026-02-20T17:59:34.355 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('6' '0' '7' '9') 2026-02-20T17:59:34.355 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-20T17:59:34.355 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=3 2026-02-20T17:59:34.355 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 9 2026-02-20T17:59:34.602 INFO:tasks.workunit.client.0.trial195.stderr:osd.9 is already out. 2026-02-20T17:59:34.610 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-20T17:59:39.612 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds ecpool SOMETHING 2026-02-20T17:59:39.612 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-20T17:59:39.612 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<9\>' 2026-02-20T17:59:39.613 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-20T17:59:39.613 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool SOMETHING 2026-02-20T17:59:39.613 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T17:59:39.841 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=6 2026-02-20T17:59:39.841 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T17:59:39.841 INFO:tasks.workunit.client.0.trial195.stderr:7 2026-02-20T17:59:39.842 INFO:tasks.workunit.client.0.trial195.stderr:5' 2026-02-20T17:59:39.842 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 6 0 7 5 2026-02-20T17:59:39.842 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool ecpool get SOMETHING td/test-erasure-code/COPY 2026-02-20T17:59:39.861 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-20T17:59:39.862 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 9 2026-02-20T17:59:40.143 INFO:tasks.workunit.client.0.trial195.stderr:osd.9 is already in. 2026-02-20T17:59:40.152 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-20T17:59:40.153 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:216: TEST_rados_put_get_jerasure: local poolname=pool-jerasure 2026-02-20T17:59:40.153 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:217: TEST_rados_put_get_jerasure: local profile=profile-jerasure 2026-02-20T17:59:40.153 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:219: TEST_rados_put_get_jerasure: ceph osd erasure-code-profile set profile-jerasure plugin=jerasure k=4 m=2 crush-failure-domain=osd 2026-02-20T17:59:40.455 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:223: TEST_rados_put_get_jerasure: create_pool pool-jerasure 12 12 erasure profile-jerasure 2026-02-20T17:59:40.456 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 12 12 erasure profile-jerasure 2026-02-20T17:59:40.829 INFO:tasks.workunit.client.0.trial195.stderr:pool 'pool-jerasure' already exists 2026-02-20T17:59:40.838 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-20T17:59:41.840 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:226: TEST_rados_put_get_jerasure: rados_put_get td/test-erasure-code pool-jerasure 2026-02-20T17:59:41.841 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-20T17:59:41.841 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-jerasure 2026-02-20T17:59:41.841 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-20T17:59:41.841 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-20T17:59:41.841 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-20T17:59:41.841 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-20T17:59:41.841 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-20T17:59:41.841 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-20T17:59:41.842 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-20T17:59:41.842 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-20T17:59:41.842 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-20T17:59:41.842 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-jerasure put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-20T17:59:41.861 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-jerasure get SOMETHING td/test-erasure-code/COPY 2026-02-20T17:59:41.881 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-20T17:59:41.882 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-20T17:59:41.883 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-jerasure SOMETHING 2026-02-20T17:59:41.884 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T17:59:41.884 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-20T17:59:41.884 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-20T17:59:41.884 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T17:59:42.113 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-20T17:59:42.114 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T17:59:42.114 INFO:tasks.workunit.client.0.trial195.stderr:9 2026-02-20T17:59:42.114 INFO:tasks.workunit.client.0.trial195.stderr:4 2026-02-20T17:59:42.114 INFO:tasks.workunit.client.0.trial195.stderr:3 2026-02-20T17:59:42.114 INFO:tasks.workunit.client.0.trial195.stderr:10' 2026-02-20T17:59:42.114 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 1 9 4 3 10 2026-02-20T17:59:42.114 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('5' '1' '9' '4' '3' '10') 2026-02-20T17:59:42.114 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-20T17:59:42.115 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=5 2026-02-20T17:59:42.115 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 10 2026-02-20T17:59:42.363 INFO:tasks.workunit.client.0.trial195.stderr:osd.10 is already out. 2026-02-20T17:59:42.372 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-20T17:59:47.374 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-jerasure SOMETHING 2026-02-20T17:59:47.375 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T17:59:47.375 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<10\>' 2026-02-20T17:59:47.375 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-20T17:59:47.375 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-20T17:59:47.376 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T17:59:47.602 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-20T17:59:47.603 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T17:59:47.603 INFO:tasks.workunit.client.0.trial195.stderr:9 2026-02-20T17:59:47.603 INFO:tasks.workunit.client.0.trial195.stderr:4 2026-02-20T17:59:47.603 INFO:tasks.workunit.client.0.trial195.stderr:3 2026-02-20T17:59:47.603 INFO:tasks.workunit.client.0.trial195.stderr:6' 2026-02-20T17:59:47.603 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 1 9 4 3 6 2026-02-20T17:59:47.603 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-jerasure get SOMETHING td/test-erasure-code/COPY 2026-02-20T17:59:47.621 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-20T17:59:47.622 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 10 2026-02-20T17:59:47.855 INFO:tasks.workunit.client.0.trial195.stderr:osd.10 is already in. 2026-02-20T17:59:47.864 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-20T17:59:47.866 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:227: TEST_rados_put_get_jerasure: rados_osds_out_in td/test-erasure-code pool-jerasure 2026-02-20T17:59:47.867 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:104: rados_osds_out_in: local dir=td/test-erasure-code 2026-02-20T17:59:47.867 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:105: rados_osds_out_in: local poolname=pool-jerasure 2026-02-20T17:59:47.867 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:106: rados_osds_out_in: local objname=SOMETHING 2026-02-20T17:59:47.867 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:109: rados_osds_out_in: for marker in FFFF GGGG HHHH IIII 2026-02-20T17:59:47.867 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:110: rados_osds_out_in: printf '%*s' 1024 FFFF 2026-02-20T17:59:47.867 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:109: rados_osds_out_in: for marker in FFFF GGGG HHHH IIII 2026-02-20T17:59:47.867 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:110: rados_osds_out_in: printf '%*s' 1024 GGGG 2026-02-20T17:59:47.867 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:109: rados_osds_out_in: for marker in FFFF GGGG HHHH IIII 2026-02-20T17:59:47.867 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:110: rados_osds_out_in: printf '%*s' 1024 HHHH 2026-02-20T17:59:47.867 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:109: rados_osds_out_in: for marker in FFFF GGGG HHHH IIII 2026-02-20T17:59:47.867 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:110: rados_osds_out_in: printf '%*s' 1024 IIII 2026-02-20T17:59:47.868 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:116: rados_osds_out_in: rados --pool pool-jerasure put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-20T17:59:47.903 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:117: rados_osds_out_in: rados --pool pool-jerasure get SOMETHING td/test-erasure-code/COPY 2026-02-20T17:59:47.923 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:118: rados_osds_out_in: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-20T17:59:47.924 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:119: rados_osds_out_in: rm td/test-erasure-code/COPY 2026-02-20T17:59:47.925 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:127: rados_osds_out_in: wait_for_clean 2026-02-20T17:59:47.925 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T17:59:47.925 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T17:59:47.925 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T17:59:47.925 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T17:59:47.926 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T17:59:47.926 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T17:59:47.926 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T17:59:47.926 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T17:59:47.926 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T17:59:48.007 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T17:59:48.008 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T17:59:48.008 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T17:59:48.008 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T17:59:48.008 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T17:59:48.008 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T17:59:48.241 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T17:59:48.241 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T17:59:48.241 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T17:59:48.242 INFO:tasks.workunit.client.0.trial195.stderr:3 2026-02-20T17:59:48.242 INFO:tasks.workunit.client.0.trial195.stderr:4 2026-02-20T17:59:48.242 INFO:tasks.workunit.client.0.trial195.stderr:5 2026-02-20T17:59:48.242 INFO:tasks.workunit.client.0.trial195.stderr:6 2026-02-20T17:59:48.242 INFO:tasks.workunit.client.0.trial195.stderr:7 2026-02-20T17:59:48.242 INFO:tasks.workunit.client.0.trial195.stderr:8 2026-02-20T17:59:48.242 INFO:tasks.workunit.client.0.trial195.stderr:9 2026-02-20T17:59:48.242 INFO:tasks.workunit.client.0.trial195.stderr:10' 2026-02-20T17:59:48.242 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T17:59:48.242 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:48.242 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T17:59:48.306 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836514 2026-02-20T17:59:48.306 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836514 2026-02-20T17:59:48.306 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836514' 2026-02-20T17:59:48.306 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:48.306 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T17:59:48.371 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672995 2026-02-20T17:59:48.371 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672995 2026-02-20T17:59:48.371 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836514 1-42949672995' 2026-02-20T17:59:48.371 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:48.371 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T17:59:48.437 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509473 2026-02-20T17:59:48.437 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509473 2026-02-20T17:59:48.437 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836514 1-42949672995 2-64424509473' 2026-02-20T17:59:48.437 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:48.437 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T17:59:48.503 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=317827579926 2026-02-20T17:59:48.503 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 317827579926 2026-02-20T17:59:48.503 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836514 1-42949672995 2-64424509473 3-317827579926' 2026-02-20T17:59:48.503 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:48.503 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-20T17:59:48.565 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=103079215136 2026-02-20T17:59:48.565 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 103079215136 2026-02-20T17:59:48.565 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836514 1-42949672995 2-64424509473 3-317827579926 4-103079215136' 2026-02-20T17:59:48.565 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:48.566 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-20T17:59:48.630 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=339302416402 2026-02-20T17:59:48.631 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 339302416402 2026-02-20T17:59:48.631 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836514 1-42949672995 2-64424509473 3-317827579926 4-103079215136 5-339302416402' 2026-02-20T17:59:48.631 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:48.631 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-20T17:59:48.695 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=386547056654 2026-02-20T17:59:48.695 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 386547056654 2026-02-20T17:59:48.695 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836514 1-42949672995 2-64424509473 3-317827579926 4-103079215136 5-339302416402 6-386547056654' 2026-02-20T17:59:48.695 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:48.695 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-20T17:59:48.759 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757279 2026-02-20T17:59:48.760 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757279 2026-02-20T17:59:48.760 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836514 1-42949672995 2-64424509473 3-317827579926 4-103079215136 5-339302416402 6-386547056654 7-163208757279' 2026-02-20T17:59:48.760 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:48.760 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-20T17:59:48.823 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=180388626462 2026-02-20T17:59:48.823 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 180388626462 2026-02-20T17:59:48.823 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836514 1-42949672995 2-64424509473 3-317827579926 4-103079215136 5-339302416402 6-386547056654 7-163208757279 8-180388626462' 2026-02-20T17:59:48.823 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:48.823 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-20T17:59:48.886 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=408021893131 2026-02-20T17:59:48.886 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 408021893131 2026-02-20T17:59:48.886 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836514 1-42949672995 2-64424509473 3-317827579926 4-103079215136 5-339302416402 6-386547056654 7-163208757279 8-180388626462 9-408021893131' 2026-02-20T17:59:48.887 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:48.887 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.10 flush_pg_stats 2026-02-20T17:59:48.951 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=214748364829 2026-02-20T17:59:48.951 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 214748364829 2026-02-20T17:59:48.951 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836514 1-42949672995 2-64424509473 3-317827579926 4-103079215136 5-339302416402 6-386547056654 7-163208757279 8-180388626462 9-408021893131 10-214748364829' 2026-02-20T17:59:48.952 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:48.952 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836514 2026-02-20T17:59:48.952 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:48.952 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T17:59:48.953 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836514 2026-02-20T17:59:48.953 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:48.954 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836514 2026-02-20T17:59:48.954 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.0 seq 21474836514 2026-02-20T17:59:48.954 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836514' 2026-02-20T17:59:48.955 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T17:59:49.179 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836512 -lt 21474836514 2026-02-20T17:59:49.179 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T17:59:50.180 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T17:59:50.180 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T17:59:50.404 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836514 -lt 21474836514 2026-02-20T17:59:50.404 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:50.405 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672995 2026-02-20T17:59:50.405 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:50.405 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T17:59:50.406 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672995 2026-02-20T17:59:50.406 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:50.407 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672995 2026-02-20T17:59:50.407 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672995' 2026-02-20T17:59:50.408 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.1 seq 42949672995 2026-02-20T17:59:50.408 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T17:59:50.632 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672995 -lt 42949672995 2026-02-20T17:59:50.632 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:50.632 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509473 2026-02-20T17:59:50.633 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:50.633 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T17:59:50.634 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509473 2026-02-20T17:59:50.634 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:50.635 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509473 2026-02-20T17:59:50.635 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509473' 2026-02-20T17:59:50.636 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.2 seq 64424509473 2026-02-20T17:59:50.636 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T17:59:50.859 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509473 -lt 64424509473 2026-02-20T17:59:50.859 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:50.860 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-317827579926 2026-02-20T17:59:50.860 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:50.860 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T17:59:50.861 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-317827579926 2026-02-20T17:59:50.861 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:50.862 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=317827579926 2026-02-20T17:59:50.862 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 317827579926' 2026-02-20T17:59:50.863 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.3 seq 317827579926 2026-02-20T17:59:50.863 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T17:59:51.089 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 317827579926 -lt 317827579926 2026-02-20T17:59:51.090 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:51.090 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-103079215136 2026-02-20T17:59:51.090 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:51.091 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-20T17:59:51.092 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-103079215136 2026-02-20T17:59:51.092 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:51.093 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=103079215136 2026-02-20T17:59:51.093 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 103079215136' 2026-02-20T17:59:51.093 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.4 seq 103079215136 2026-02-20T17:59:51.094 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-20T17:59:51.317 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 103079215136 -lt 103079215136 2026-02-20T17:59:51.318 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:51.318 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-339302416402 2026-02-20T17:59:51.318 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:51.319 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-20T17:59:51.319 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-339302416402 2026-02-20T17:59:51.320 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:51.321 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=339302416402 2026-02-20T17:59:51.321 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 339302416402' 2026-02-20T17:59:51.321 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.5 seq 339302416402 2026-02-20T17:59:51.321 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-20T17:59:51.546 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 339302416402 -lt 339302416402 2026-02-20T17:59:51.547 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:51.547 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-386547056654 2026-02-20T17:59:51.548 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:51.548 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-20T17:59:51.549 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-386547056654 2026-02-20T17:59:51.549 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:51.550 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=386547056654 2026-02-20T17:59:51.550 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.6 seq 386547056654 2026-02-20T17:59:51.551 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 386547056654' 2026-02-20T17:59:51.551 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-20T17:59:51.774 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 386547056654 -lt 386547056654 2026-02-20T17:59:51.774 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:51.774 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-163208757279 2026-02-20T17:59:51.775 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:51.775 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-20T17:59:51.776 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-163208757279 2026-02-20T17:59:51.776 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:51.777 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=163208757279 2026-02-20T17:59:51.777 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.7 seq 163208757279 2026-02-20T17:59:51.778 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 163208757279' 2026-02-20T17:59:51.778 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-20T17:59:52.003 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 163208757279 -lt 163208757279 2026-02-20T17:59:52.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:52.004 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-180388626462 2026-02-20T17:59:52.004 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:52.005 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-20T17:59:52.006 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-180388626462 2026-02-20T17:59:52.006 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:52.007 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=180388626462 2026-02-20T17:59:52.007 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 180388626462' 2026-02-20T17:59:52.007 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.8 seq 180388626462 2026-02-20T17:59:52.007 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-20T17:59:52.230 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 180388626463 -lt 180388626462 2026-02-20T17:59:52.230 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:52.231 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-408021893131 2026-02-20T17:59:52.231 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:52.232 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-20T17:59:52.233 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-408021893131 2026-02-20T17:59:52.233 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:52.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=408021893131 2026-02-20T17:59:52.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 408021893131' 2026-02-20T17:59:52.234 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.9 seq 408021893131 2026-02-20T17:59:52.235 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-20T17:59:52.463 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 408021893131 -lt 408021893131 2026-02-20T17:59:52.464 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:52.464 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-214748364829 2026-02-20T17:59:52.464 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:52.465 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=10 2026-02-20T17:59:52.466 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 10-214748364829 2026-02-20T17:59:52.466 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:52.467 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=214748364829 2026-02-20T17:59:52.467 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.10 seq 214748364829' 2026-02-20T17:59:52.468 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.10 seq 214748364829 2026-02-20T17:59:52.468 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-20T17:59:52.691 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 214748364830 -lt 214748364829 2026-02-20T17:59:52.692 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T17:59:52.692 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T17:59:52.693 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T17:59:53.001 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-20T17:59:53.001 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T17:59:53.001 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T17:59:53.002 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T17:59:53.002 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T17:59:53.002 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T17:59:53.002 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T17:59:53.002 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T17:59:53.226 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-20T17:59:53.227 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T17:59:53.227 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T17:59:53.227 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T17:59:53.553 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-20T17:59:53.553 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T17:59:53.553 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T17:59:53.554 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:128: rados_osds_out_in: get_osds pool-jerasure SOMETHING 2026-02-20T17:59:53.554 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T17:59:53.554 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-20T17:59:53.554 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-20T17:59:53.554 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T17:59:53.784 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-20T17:59:53.785 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T17:59:53.785 INFO:tasks.workunit.client.0.trial195.stderr:9 2026-02-20T17:59:53.785 INFO:tasks.workunit.client.0.trial195.stderr:4 2026-02-20T17:59:53.785 INFO:tasks.workunit.client.0.trial195.stderr:3 2026-02-20T17:59:53.785 INFO:tasks.workunit.client.0.trial195.stderr:10' 2026-02-20T17:59:53.785 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 1 9 4 3 10 2026-02-20T17:59:53.785 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:128: rados_osds_out_in: local 'osds_list=5 1 9 4 3 10' 2026-02-20T17:59:53.785 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:129: rados_osds_out_in: osds=('5' '1' '9' '4' '3' '10') 2026-02-20T17:59:53.785 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:129: rados_osds_out_in: local -a osds 2026-02-20T17:59:53.785 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:130: rados_osds_out_in: for osd in 0 1 2026-02-20T17:59:53.785 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:131: rados_osds_out_in: ceph osd out 5 2026-02-20T17:59:54.036 INFO:tasks.workunit.client.0.trial195.stderr:osd.5 is already out. 2026-02-20T17:59:54.046 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:130: rados_osds_out_in: for osd in 0 1 2026-02-20T17:59:54.046 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:131: rados_osds_out_in: ceph osd out 1 2026-02-20T17:59:54.339 INFO:tasks.workunit.client.0.trial195.stderr:osd.1 is already out. 2026-02-20T17:59:54.349 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:133: rados_osds_out_in: wait_for_clean 2026-02-20T17:59:54.349 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T17:59:54.349 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T17:59:54.349 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T17:59:54.350 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T17:59:54.351 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T17:59:54.351 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T17:59:54.351 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T17:59:54.351 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T17:59:54.351 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T17:59:54.431 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T17:59:54.431 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T17:59:54.431 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T17:59:54.431 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T17:59:54.431 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T17:59:54.431 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T17:59:54.654 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T17:59:54.655 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T17:59:54.655 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T17:59:54.655 INFO:tasks.workunit.client.0.trial195.stderr:3 2026-02-20T17:59:54.655 INFO:tasks.workunit.client.0.trial195.stderr:4 2026-02-20T17:59:54.655 INFO:tasks.workunit.client.0.trial195.stderr:5 2026-02-20T17:59:54.655 INFO:tasks.workunit.client.0.trial195.stderr:6 2026-02-20T17:59:54.655 INFO:tasks.workunit.client.0.trial195.stderr:7 2026-02-20T17:59:54.655 INFO:tasks.workunit.client.0.trial195.stderr:8 2026-02-20T17:59:54.655 INFO:tasks.workunit.client.0.trial195.stderr:9 2026-02-20T17:59:54.655 INFO:tasks.workunit.client.0.trial195.stderr:10' 2026-02-20T17:59:54.656 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T17:59:54.656 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:54.656 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T17:59:54.721 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836517 2026-02-20T17:59:54.721 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836517 2026-02-20T17:59:54.721 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836517' 2026-02-20T17:59:54.721 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:54.721 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T17:59:54.785 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672998 2026-02-20T17:59:54.785 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672998 2026-02-20T17:59:54.785 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836517 1-42949672998' 2026-02-20T17:59:54.785 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:54.785 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T17:59:54.851 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509476 2026-02-20T17:59:54.851 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509476 2026-02-20T17:59:54.852 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836517 1-42949672998 2-64424509476' 2026-02-20T17:59:54.852 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:54.852 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T17:59:54.915 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=317827579929 2026-02-20T17:59:54.915 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 317827579929 2026-02-20T17:59:54.915 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836517 1-42949672998 2-64424509476 3-317827579929' 2026-02-20T17:59:54.915 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:54.915 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-20T17:59:54.979 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=103079215139 2026-02-20T17:59:54.979 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 103079215139 2026-02-20T17:59:54.979 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836517 1-42949672998 2-64424509476 3-317827579929 4-103079215139' 2026-02-20T17:59:54.979 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:54.979 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-20T17:59:55.044 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=339302416405 2026-02-20T17:59:55.044 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 339302416405 2026-02-20T17:59:55.044 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836517 1-42949672998 2-64424509476 3-317827579929 4-103079215139 5-339302416405' 2026-02-20T17:59:55.044 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:55.044 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-20T17:59:55.108 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=386547056657 2026-02-20T17:59:55.108 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 386547056657 2026-02-20T17:59:55.108 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836517 1-42949672998 2-64424509476 3-317827579929 4-103079215139 5-339302416405 6-386547056657' 2026-02-20T17:59:55.109 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:55.109 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-20T17:59:55.173 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757282 2026-02-20T17:59:55.174 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757282 2026-02-20T17:59:55.174 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836517 1-42949672998 2-64424509476 3-317827579929 4-103079215139 5-339302416405 6-386547056657 7-163208757282' 2026-02-20T17:59:55.174 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:55.174 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-20T17:59:55.241 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=180388626465 2026-02-20T17:59:55.242 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 180388626465 2026-02-20T17:59:55.242 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836517 1-42949672998 2-64424509476 3-317827579929 4-103079215139 5-339302416405 6-386547056657 7-163208757282 8-180388626465' 2026-02-20T17:59:55.242 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:55.242 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-20T17:59:55.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=408021893134 2026-02-20T17:59:55.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 408021893134 2026-02-20T17:59:55.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836517 1-42949672998 2-64424509476 3-317827579929 4-103079215139 5-339302416405 6-386547056657 7-163208757282 8-180388626465 9-408021893134' 2026-02-20T17:59:55.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T17:59:55.308 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.10 flush_pg_stats 2026-02-20T17:59:55.374 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=214748364832 2026-02-20T17:59:55.374 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 214748364832 2026-02-20T17:59:55.374 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836517 1-42949672998 2-64424509476 3-317827579929 4-103079215139 5-339302416405 6-386547056657 7-163208757282 8-180388626465 9-408021893134 10-214748364832' 2026-02-20T17:59:55.374 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:55.375 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836517 2026-02-20T17:59:55.375 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:55.375 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T17:59:55.376 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836517 2026-02-20T17:59:55.376 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:55.377 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836517 2026-02-20T17:59:55.377 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836517' 2026-02-20T17:59:55.378 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.0 seq 21474836517 2026-02-20T17:59:55.378 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T17:59:55.599 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836515 -lt 21474836517 2026-02-20T17:59:55.599 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T17:59:56.601 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T17:59:56.601 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T17:59:56.828 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836518 -lt 21474836517 2026-02-20T17:59:56.828 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:56.828 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672998 2026-02-20T17:59:56.829 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:56.829 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T17:59:56.830 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672998 2026-02-20T17:59:56.830 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:56.831 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672998 2026-02-20T17:59:56.831 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672998' 2026-02-20T17:59:56.831 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.1 seq 42949672998 2026-02-20T17:59:56.832 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T17:59:57.059 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672998 -lt 42949672998 2026-02-20T17:59:57.060 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:57.060 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509476 2026-02-20T17:59:57.060 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:57.061 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T17:59:57.062 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509476 2026-02-20T17:59:57.062 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:57.063 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509476 2026-02-20T17:59:57.063 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.2 seq 64424509476 2026-02-20T17:59:57.063 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509476' 2026-02-20T17:59:57.064 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T17:59:57.291 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509477 -lt 64424509476 2026-02-20T17:59:57.291 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:57.292 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-317827579929 2026-02-20T17:59:57.292 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:57.292 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T17:59:57.293 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-317827579929 2026-02-20T17:59:57.293 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:57.294 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=317827579929 2026-02-20T17:59:57.294 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 317827579929' 2026-02-20T17:59:57.295 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.3 seq 317827579929 2026-02-20T17:59:57.295 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T17:59:57.521 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 317827579929 -lt 317827579929 2026-02-20T17:59:57.521 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:57.522 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-103079215139 2026-02-20T17:59:57.522 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:57.523 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-20T17:59:57.523 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-103079215139 2026-02-20T17:59:57.524 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:57.525 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=103079215139 2026-02-20T17:59:57.525 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 103079215139' 2026-02-20T17:59:57.525 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.4 seq 103079215139 2026-02-20T17:59:57.525 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-20T17:59:57.748 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 103079215140 -lt 103079215139 2026-02-20T17:59:57.748 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:57.749 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-339302416405 2026-02-20T17:59:57.749 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:57.750 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-20T17:59:57.750 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-339302416405 2026-02-20T17:59:57.751 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:57.752 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=339302416405 2026-02-20T17:59:57.752 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 339302416405' 2026-02-20T17:59:57.752 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.5 seq 339302416405 2026-02-20T17:59:57.752 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-20T17:59:57.976 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 339302416406 -lt 339302416405 2026-02-20T17:59:57.976 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:57.977 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-386547056657 2026-02-20T17:59:57.977 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:57.978 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-20T17:59:57.978 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-386547056657 2026-02-20T17:59:57.979 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:57.979 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=386547056657 2026-02-20T17:59:57.980 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.6 seq 386547056657 2026-02-20T17:59:57.980 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 386547056657' 2026-02-20T17:59:57.981 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-20T17:59:58.209 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 386547056658 -lt 386547056657 2026-02-20T17:59:58.209 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:58.210 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-163208757282 2026-02-20T17:59:58.210 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:58.211 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-20T17:59:58.211 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-163208757282 2026-02-20T17:59:58.211 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:58.213 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=163208757282 2026-02-20T17:59:58.213 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 163208757282' 2026-02-20T17:59:58.213 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.7 seq 163208757282 2026-02-20T17:59:58.213 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-20T17:59:58.439 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 163208757282 -lt 163208757282 2026-02-20T17:59:58.439 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:58.440 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:58.440 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-180388626465 2026-02-20T17:59:58.440 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-20T17:59:58.441 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-180388626465 2026-02-20T17:59:58.442 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:58.443 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=180388626465 2026-02-20T17:59:58.443 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.8 seq 180388626465 2026-02-20T17:59:58.443 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 180388626465' 2026-02-20T17:59:58.443 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-20T17:59:58.665 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 180388626466 -lt 180388626465 2026-02-20T17:59:58.666 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:58.666 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-408021893134 2026-02-20T17:59:58.667 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:58.667 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-20T17:59:58.668 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-408021893134 2026-02-20T17:59:58.668 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:58.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=408021893134 2026-02-20T17:59:58.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 408021893134' 2026-02-20T17:59:58.669 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.9 seq 408021893134 2026-02-20T17:59:58.669 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-20T17:59:58.895 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 408021893134 -lt 408021893134 2026-02-20T17:59:58.895 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T17:59:58.896 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-214748364832 2026-02-20T17:59:58.896 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T17:59:58.897 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=10 2026-02-20T17:59:58.897 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 10-214748364832 2026-02-20T17:59:58.898 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T17:59:58.898 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=214748364832 2026-02-20T17:59:58.899 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.10 seq 214748364832' 2026-02-20T17:59:58.899 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.10 seq 214748364832 2026-02-20T17:59:58.899 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-20T17:59:59.122 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 214748364833 -lt 214748364832 2026-02-20T17:59:59.122 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T17:59:59.123 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T17:59:59.123 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T17:59:59.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-20T17:59:59.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T17:59:59.438 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T17:59:59.438 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T17:59:59.438 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T17:59:59.438 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T17:59:59.438 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T17:59:59.438 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T17:59:59.660 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-20T17:59:59.661 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T17:59:59.661 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T17:59:59.661 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T17:59:59.981 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-20T17:59:59.981 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T17:59:59.981 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T17:59:59.981 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:137: rados_osds_out_in: for osd in 0 1 2026-02-20T17:59:59.981 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:138: rados_osds_out_in: get_osds pool-jerasure SOMETHING 2026-02-20T17:59:59.981 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T17:59:59.981 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-20T17:59:59.982 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:138: rados_osds_out_in: grep '\<5\>' 2026-02-20T17:59:59.982 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-20T17:59:59.982 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:00:00.209 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=2 2026-02-20T18:00:00.209 INFO:tasks.workunit.client.0.trial195.stderr:7 2026-02-20T18:00:00.209 INFO:tasks.workunit.client.0.trial195.stderr:9 2026-02-20T18:00:00.209 INFO:tasks.workunit.client.0.trial195.stderr:4 2026-02-20T18:00:00.209 INFO:tasks.workunit.client.0.trial195.stderr:3 2026-02-20T18:00:00.210 INFO:tasks.workunit.client.0.trial195.stderr:10' 2026-02-20T18:00:00.210 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 2 7 9 4 3 10 2026-02-20T18:00:00.210 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:137: rados_osds_out_in: for osd in 0 1 2026-02-20T18:00:00.211 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:138: rados_osds_out_in: get_osds pool-jerasure SOMETHING 2026-02-20T18:00:00.211 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:00:00.211 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-20T18:00:00.211 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:138: rados_osds_out_in: grep '\<1\>' 2026-02-20T18:00:00.211 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-20T18:00:00.211 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:00:00.437 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=2 2026-02-20T18:00:00.437 INFO:tasks.workunit.client.0.trial195.stderr:7 2026-02-20T18:00:00.437 INFO:tasks.workunit.client.0.trial195.stderr:9 2026-02-20T18:00:00.437 INFO:tasks.workunit.client.0.trial195.stderr:4 2026-02-20T18:00:00.437 INFO:tasks.workunit.client.0.trial195.stderr:3 2026-02-20T18:00:00.437 INFO:tasks.workunit.client.0.trial195.stderr:10' 2026-02-20T18:00:00.437 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 2 7 9 4 3 10 2026-02-20T18:00:00.437 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:140: rados_osds_out_in: rados --pool pool-jerasure get SOMETHING td/test-erasure-code/COPY 2026-02-20T18:00:00.458 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:141: rados_osds_out_in: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-20T18:00:00.458 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:147: rados_osds_out_in: for osd in 0 1 2026-02-20T18:00:00.459 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:148: rados_osds_out_in: ceph osd in 5 2026-02-20T18:00:00.738 INFO:tasks.workunit.client.0.trial195.stderr:osd.5 is already in. 2026-02-20T18:00:00.747 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:147: rados_osds_out_in: for osd in 0 1 2026-02-20T18:00:00.747 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:148: rados_osds_out_in: ceph osd in 1 2026-02-20T18:00:01.033 INFO:tasks.workunit.client.0.trial195.stderr:osd.1 is already in. 2026-02-20T18:00:01.042 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:150: rados_osds_out_in: wait_for_clean 2026-02-20T18:00:01.042 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:00:01.042 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:00:01.042 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:00:01.043 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:00:01.043 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:00:01.043 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:00:01.043 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:00:01.043 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:00:01.043 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:00:01.124 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:00:01.124 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:00:01.124 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:00:01.124 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:00:01.124 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:00:01.124 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:00:01.359 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:00:01.360 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:00:01.360 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:00:01.360 INFO:tasks.workunit.client.0.trial195.stderr:3 2026-02-20T18:00:01.360 INFO:tasks.workunit.client.0.trial195.stderr:4 2026-02-20T18:00:01.360 INFO:tasks.workunit.client.0.trial195.stderr:5 2026-02-20T18:00:01.360 INFO:tasks.workunit.client.0.trial195.stderr:6 2026-02-20T18:00:01.360 INFO:tasks.workunit.client.0.trial195.stderr:7 2026-02-20T18:00:01.360 INFO:tasks.workunit.client.0.trial195.stderr:8 2026-02-20T18:00:01.360 INFO:tasks.workunit.client.0.trial195.stderr:9 2026-02-20T18:00:01.360 INFO:tasks.workunit.client.0.trial195.stderr:10' 2026-02-20T18:00:01.361 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:00:01.361 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:00:01.361 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:00:01.427 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836521 2026-02-20T18:00:01.427 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836521 2026-02-20T18:00:01.428 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836521' 2026-02-20T18:00:01.428 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:00:01.428 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:00:01.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949673001 2026-02-20T18:00:01.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949673001 2026-02-20T18:00:01.493 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836521 1-42949673001' 2026-02-20T18:00:01.493 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:00:01.493 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:00:01.555 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509480 2026-02-20T18:00:01.555 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509480 2026-02-20T18:00:01.556 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836521 1-42949673001 2-64424509480' 2026-02-20T18:00:01.556 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:00:01.556 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:00:01.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=317827579932 2026-02-20T18:00:01.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 317827579932 2026-02-20T18:00:01.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836521 1-42949673001 2-64424509480 3-317827579932' 2026-02-20T18:00:01.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:00:01.620 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-20T18:00:01.685 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=103079215143 2026-02-20T18:00:01.685 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 103079215143 2026-02-20T18:00:01.685 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836521 1-42949673001 2-64424509480 3-317827579932 4-103079215143' 2026-02-20T18:00:01.685 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:00:01.685 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-20T18:00:01.747 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=339302416409 2026-02-20T18:00:01.747 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 339302416409 2026-02-20T18:00:01.748 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836521 1-42949673001 2-64424509480 3-317827579932 4-103079215143 5-339302416409' 2026-02-20T18:00:01.748 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:00:01.748 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-20T18:00:01.812 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=386547056661 2026-02-20T18:00:01.812 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 386547056661 2026-02-20T18:00:01.813 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836521 1-42949673001 2-64424509480 3-317827579932 4-103079215143 5-339302416409 6-386547056661' 2026-02-20T18:00:01.813 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:00:01.813 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-20T18:00:01.877 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757285 2026-02-20T18:00:01.877 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757285 2026-02-20T18:00:01.877 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836521 1-42949673001 2-64424509480 3-317827579932 4-103079215143 5-339302416409 6-386547056661 7-163208757285' 2026-02-20T18:00:01.877 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:00:01.877 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-20T18:00:01.942 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=180388626469 2026-02-20T18:00:01.942 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 180388626469 2026-02-20T18:00:01.942 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836521 1-42949673001 2-64424509480 3-317827579932 4-103079215143 5-339302416409 6-386547056661 7-163208757285 8-180388626469' 2026-02-20T18:00:01.942 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:00:01.942 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-20T18:00:02.005 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=408021893137 2026-02-20T18:00:02.006 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 408021893137 2026-02-20T18:00:02.006 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836521 1-42949673001 2-64424509480 3-317827579932 4-103079215143 5-339302416409 6-386547056661 7-163208757285 8-180388626469 9-408021893137' 2026-02-20T18:00:02.006 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:00:02.006 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.10 flush_pg_stats 2026-02-20T18:00:02.070 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=214748364836 2026-02-20T18:00:02.070 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 214748364836 2026-02-20T18:00:02.070 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836521 1-42949673001 2-64424509480 3-317827579932 4-103079215143 5-339302416409 6-386547056661 7-163208757285 8-180388626469 9-408021893137 10-214748364836' 2026-02-20T18:00:02.070 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:00:02.071 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836521 2026-02-20T18:00:02.071 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:00:02.071 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:00:02.072 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836521 2026-02-20T18:00:02.072 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:00:02.073 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836521 2026-02-20T18:00:02.073 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836521' 2026-02-20T18:00:02.074 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.0 seq 21474836521 2026-02-20T18:00:02.074 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:00:02.298 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836521 -lt 21474836521 2026-02-20T18:00:02.299 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:00:02.299 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949673001 2026-02-20T18:00:02.299 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:00:02.300 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:00:02.301 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949673001 2026-02-20T18:00:02.301 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:00:02.302 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949673001 2026-02-20T18:00:02.302 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949673001' 2026-02-20T18:00:02.302 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.1 seq 42949673001 2026-02-20T18:00:02.302 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:00:02.523 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949673001 -lt 42949673001 2026-02-20T18:00:02.523 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:00:02.524 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509480 2026-02-20T18:00:02.524 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:00:02.524 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:00:02.525 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509480 2026-02-20T18:00:02.525 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:00:02.527 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509480 2026-02-20T18:00:02.527 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.2 seq 64424509480 2026-02-20T18:00:02.528 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509480' 2026-02-20T18:00:02.528 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:00:02.756 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509480 -lt 64424509480 2026-02-20T18:00:02.756 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:00:02.756 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-317827579932 2026-02-20T18:00:02.756 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:00:02.757 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:00:02.757 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-317827579932 2026-02-20T18:00:02.758 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:00:02.758 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=317827579932 2026-02-20T18:00:02.758 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 317827579932' 2026-02-20T18:00:02.759 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.3 seq 317827579932 2026-02-20T18:00:02.759 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:00:02.982 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 317827579932 -lt 317827579932 2026-02-20T18:00:02.982 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:00:02.983 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-103079215143 2026-02-20T18:00:02.983 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:00:02.983 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-20T18:00:02.984 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-103079215143 2026-02-20T18:00:02.984 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:00:02.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=103079215143 2026-02-20T18:00:02.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 103079215143' 2026-02-20T18:00:02.986 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.4 seq 103079215143 2026-02-20T18:00:02.986 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-20T18:00:03.211 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 103079215143 -lt 103079215143 2026-02-20T18:00:03.211 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:00:03.211 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-339302416409 2026-02-20T18:00:03.212 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:00:03.212 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-20T18:00:03.213 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-339302416409 2026-02-20T18:00:03.213 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:00:03.214 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=339302416409 2026-02-20T18:00:03.214 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 339302416409' 2026-02-20T18:00:03.215 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.5 seq 339302416409 2026-02-20T18:00:03.215 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-20T18:00:03.440 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 339302416409 -lt 339302416409 2026-02-20T18:00:03.440 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:00:03.440 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-386547056661 2026-02-20T18:00:03.441 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:00:03.441 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-20T18:00:03.442 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-386547056661 2026-02-20T18:00:03.442 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:00:03.443 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=386547056661 2026-02-20T18:00:03.443 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.6 seq 386547056661 2026-02-20T18:00:03.444 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 386547056661' 2026-02-20T18:00:03.444 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-20T18:00:03.673 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 386547056661 -lt 386547056661 2026-02-20T18:00:03.673 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:00:03.673 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-163208757285 2026-02-20T18:00:03.673 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:00:03.674 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-20T18:00:03.675 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-163208757285 2026-02-20T18:00:03.675 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:00:03.676 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=163208757285 2026-02-20T18:00:03.676 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 163208757285' 2026-02-20T18:00:03.676 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.7 seq 163208757285 2026-02-20T18:00:03.677 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-20T18:00:03.900 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 163208757285 -lt 163208757285 2026-02-20T18:00:03.900 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:00:03.901 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-180388626469 2026-02-20T18:00:03.901 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:00:03.901 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-20T18:00:03.902 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-180388626469 2026-02-20T18:00:03.902 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:00:03.903 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=180388626469 2026-02-20T18:00:03.903 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 180388626469' 2026-02-20T18:00:03.904 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.8 seq 180388626469 2026-02-20T18:00:03.904 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-20T18:00:04.130 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 180388626469 -lt 180388626469 2026-02-20T18:00:04.130 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:00:04.131 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-408021893137 2026-02-20T18:00:04.131 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:00:04.131 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-20T18:00:04.132 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-408021893137 2026-02-20T18:00:04.132 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:00:04.133 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=408021893137 2026-02-20T18:00:04.133 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 408021893137' 2026-02-20T18:00:04.133 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.9 seq 408021893137 2026-02-20T18:00:04.134 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-20T18:00:04.356 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 408021893138 -lt 408021893137 2026-02-20T18:00:04.356 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:00:04.357 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-214748364836 2026-02-20T18:00:04.357 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:00:04.357 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=10 2026-02-20T18:00:04.358 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 10-214748364836 2026-02-20T18:00:04.358 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:00:04.359 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=214748364836 2026-02-20T18:00:04.359 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.10 seq 214748364836' 2026-02-20T18:00:04.359 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.10 seq 214748364836 2026-02-20T18:00:04.360 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-20T18:00:04.585 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 214748364836 -lt 214748364836 2026-02-20T18:00:04.586 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:00:04.586 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:00:04.586 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:00:04.901 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-20T18:00:04.901 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:00:04.903 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:00:04.903 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:00:04.903 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:00:04.903 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:00:04.903 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:00:04.903 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:00:05.123 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-20T18:00:05.123 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:00:05.123 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:00:05.123 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:00:05.441 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-20T18:00:05.442 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:00:05.442 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:00:05.443 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:151: rados_osds_out_in: get_osds pool-jerasure SOMETHING 2026-02-20T18:00:05.443 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:00:05.443 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-20T18:00:05.443 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-20T18:00:05.443 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:00:05.673 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-20T18:00:05.673 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:00:05.673 INFO:tasks.workunit.client.0.trial195.stderr:9 2026-02-20T18:00:05.673 INFO:tasks.workunit.client.0.trial195.stderr:4 2026-02-20T18:00:05.673 INFO:tasks.workunit.client.0.trial195.stderr:3 2026-02-20T18:00:05.673 INFO:tasks.workunit.client.0.trial195.stderr:10' 2026-02-20T18:00:05.673 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 1 9 4 3 10 2026-02-20T18:00:05.674 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:151: rados_osds_out_in: test '5 1 9 4 3 10' = '5 1 9 4 3 10' 2026-02-20T18:00:05.674 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:152: rados_osds_out_in: rm td/test-erasure-code/ORIGINAL 2026-02-20T18:00:05.674 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:229: TEST_rados_put_get_jerasure: delete_pool pool-jerasure 2026-02-20T18:00:05.674 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-jerasure 2026-02-20T18:00:05.675 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-20T18:00:05.919 INFO:tasks.workunit.client.0.trial195.stderr:pool 'pool-jerasure' does not exist 2026-02-20T18:00:05.928 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:230: TEST_rados_put_get_jerasure: ceph osd erasure-code-profile rm profile-jerasure 2026-02-20T18:00:06.208 INFO:tasks.workunit.client.0.trial195.stderr:erasure-code-profile profile-jerasure does not exist 2026-02-20T18:00:06.217 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-20T18:00:06.218 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_lrc_advanced td/test-erasure-code 2026-02-20T18:00:06.218 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:156: TEST_rados_put_get_lrc_advanced: local dir=td/test-erasure-code 2026-02-20T18:00:06.218 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:157: TEST_rados_put_get_lrc_advanced: local poolname=pool-lrc-a 2026-02-20T18:00:06.218 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:158: TEST_rados_put_get_lrc_advanced: local profile=profile-lrc-a 2026-02-20T18:00:06.218 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:160: TEST_rados_put_get_lrc_advanced: ceph osd erasure-code-profile set profile-lrc-a plugin=lrc mapping=DD_ 'crush-steps=[ [ "chooseleaf", "osd", 0 ] ]' 'layers=[ [ "DDc", "" ] ]' 2026-02-20T18:00:06.503 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:165: TEST_rados_put_get_lrc_advanced: create_pool pool-lrc-a 12 12 erasure profile-lrc-a 2026-02-20T18:00:06.503 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-lrc-a 12 12 erasure profile-lrc-a 2026-02-20T18:00:06.877 INFO:tasks.workunit.client.0.trial195.stderr:pool 'pool-lrc-a' already exists 2026-02-20T18:00:06.886 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-20T18:00:07.889 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:168: TEST_rados_put_get_lrc_advanced: rados_put_get td/test-erasure-code pool-lrc-a 2026-02-20T18:00:07.889 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-20T18:00:07.889 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-lrc-a 2026-02-20T18:00:07.889 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-20T18:00:07.889 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-20T18:00:07.889 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-20T18:00:07.889 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-20T18:00:07.890 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-20T18:00:07.890 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-20T18:00:07.890 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-20T18:00:07.890 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-20T18:00:07.890 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-20T18:00:07.890 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-lrc-a put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-20T18:00:07.909 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-lrc-a get SOMETHING td/test-erasure-code/COPY 2026-02-20T18:00:07.928 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-20T18:00:07.928 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-20T18:00:07.930 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-lrc-a SOMETHING 2026-02-20T18:00:07.930 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-lrc-a 2026-02-20T18:00:07.930 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-20T18:00:07.931 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-lrc-a SOMETHING 2026-02-20T18:00:07.931 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:00:08.164 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-20T18:00:08.164 INFO:tasks.workunit.client.0.trial195.stderr:9 2026-02-20T18:00:08.164 INFO:tasks.workunit.client.0.trial195.stderr:3' 2026-02-20T18:00:08.164 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 9 3 2026-02-20T18:00:08.164 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('5' '9' '3') 2026-02-20T18:00:08.164 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-20T18:00:08.164 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=2 2026-02-20T18:00:08.165 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 3 2026-02-20T18:00:08.439 INFO:tasks.workunit.client.0.trial195.stderr:osd.3 is already out. 2026-02-20T18:00:08.448 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-20T18:00:13.451 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-lrc-a SOMETHING 2026-02-20T18:00:13.451 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-lrc-a 2026-02-20T18:00:13.451 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-20T18:00:13.451 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<3\>' 2026-02-20T18:00:13.452 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-lrc-a SOMETHING 2026-02-20T18:00:13.452 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:00:13.681 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-20T18:00:13.682 INFO:tasks.workunit.client.0.trial195.stderr:9 2026-02-20T18:00:13.682 INFO:tasks.workunit.client.0.trial195.stderr:7' 2026-02-20T18:00:13.682 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 9 7 2026-02-20T18:00:13.682 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-lrc-a get SOMETHING td/test-erasure-code/COPY 2026-02-20T18:00:13.701 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-20T18:00:13.702 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 3 2026-02-20T18:00:13.982 INFO:tasks.workunit.client.0.trial195.stderr:osd.3 is already in. 2026-02-20T18:00:13.991 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-20T18:00:13.992 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:170: TEST_rados_put_get_lrc_advanced: delete_pool pool-lrc-a 2026-02-20T18:00:13.992 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-lrc-a 2026-02-20T18:00:13.992 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-lrc-a pool-lrc-a --yes-i-really-really-mean-it 2026-02-20T18:00:14.270 INFO:tasks.workunit.client.0.trial195.stderr:pool 'pool-lrc-a' does not exist 2026-02-20T18:00:14.278 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:171: TEST_rados_put_get_lrc_advanced: ceph osd erasure-code-profile rm profile-lrc-a 2026-02-20T18:00:14.553 INFO:tasks.workunit.client.0.trial195.stderr:erasure-code-profile profile-lrc-a does not exist 2026-02-20T18:00:14.562 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-20T18:00:14.562 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_lrc_kml td/test-erasure-code 2026-02-20T18:00:14.562 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:175: TEST_rados_put_get_lrc_kml: local dir=td/test-erasure-code 2026-02-20T18:00:14.563 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:176: TEST_rados_put_get_lrc_kml: local poolname=pool-lrc 2026-02-20T18:00:14.563 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:177: TEST_rados_put_get_lrc_kml: local profile=profile-lrc 2026-02-20T18:00:14.563 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:179: TEST_rados_put_get_lrc_kml: ceph osd erasure-code-profile set profile-lrc plugin=lrc k=4 m=2 l=3 crush-failure-domain=osd 2026-02-20T18:00:14.849 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:183: TEST_rados_put_get_lrc_kml: create_pool pool-lrc 12 12 erasure profile-lrc 2026-02-20T18:00:14.849 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-lrc 12 12 erasure profile-lrc 2026-02-20T18:00:15.223 INFO:tasks.workunit.client.0.trial195.stderr:pool 'pool-lrc' already exists 2026-02-20T18:00:15.232 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-20T18:00:16.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:186: TEST_rados_put_get_lrc_kml: rados_put_get td/test-erasure-code pool-lrc 2026-02-20T18:00:16.235 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-20T18:00:16.235 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-lrc 2026-02-20T18:00:16.235 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-20T18:00:16.235 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-20T18:00:16.235 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-20T18:00:16.235 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-20T18:00:16.235 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-20T18:00:16.235 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-20T18:00:16.235 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-20T18:00:16.235 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-20T18:00:16.235 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-20T18:00:16.236 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-lrc put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-20T18:00:16.255 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-lrc get SOMETHING td/test-erasure-code/COPY 2026-02-20T18:00:16.274 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-20T18:00:16.275 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-20T18:00:16.277 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-lrc SOMETHING 2026-02-20T18:00:16.277 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-lrc 2026-02-20T18:00:16.277 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-20T18:00:16.277 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-lrc SOMETHING 2026-02-20T18:00:16.277 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:00:16.506 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=8 2026-02-20T18:00:16.507 INFO:tasks.workunit.client.0.trial195.stderr:6 2026-02-20T18:00:16.507 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T18:00:16.507 INFO:tasks.workunit.client.0.trial195.stderr:10 2026-02-20T18:00:16.507 INFO:tasks.workunit.client.0.trial195.stderr:5 2026-02-20T18:00:16.507 INFO:tasks.workunit.client.0.trial195.stderr:3 2026-02-20T18:00:16.507 INFO:tasks.workunit.client.0.trial195.stderr:9 2026-02-20T18:00:16.507 INFO:tasks.workunit.client.0.trial195.stderr:7' 2026-02-20T18:00:16.507 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 8 6 0 10 5 3 9 7 2026-02-20T18:00:16.507 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('8' '6' '0' '10' '5' '3' '9' '7') 2026-02-20T18:00:16.507 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-20T18:00:16.508 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=7 2026-02-20T18:00:16.508 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 7 2026-02-20T18:00:16.757 INFO:tasks.workunit.client.0.trial195.stderr:osd.7 is already out. 2026-02-20T18:00:16.767 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-20T18:00:21.769 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-lrc SOMETHING 2026-02-20T18:00:21.769 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-lrc 2026-02-20T18:00:21.769 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-20T18:00:21.770 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<7\>' 2026-02-20T18:00:21.770 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-lrc SOMETHING 2026-02-20T18:00:21.770 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:00:22.001 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=8 2026-02-20T18:00:22.001 INFO:tasks.workunit.client.0.trial195.stderr:6 2026-02-20T18:00:22.001 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T18:00:22.001 INFO:tasks.workunit.client.0.trial195.stderr:10 2026-02-20T18:00:22.001 INFO:tasks.workunit.client.0.trial195.stderr:5 2026-02-20T18:00:22.001 INFO:tasks.workunit.client.0.trial195.stderr:3 2026-02-20T18:00:22.001 INFO:tasks.workunit.client.0.trial195.stderr:9 2026-02-20T18:00:22.001 INFO:tasks.workunit.client.0.trial195.stderr:4' 2026-02-20T18:00:22.001 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 8 6 0 10 5 3 9 4 2026-02-20T18:00:22.002 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-lrc get SOMETHING td/test-erasure-code/COPY 2026-02-20T18:00:22.021 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-20T18:00:22.022 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 7 2026-02-20T18:00:22.301 INFO:tasks.workunit.client.0.trial195.stderr:osd.7 is already in. 2026-02-20T18:00:22.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-20T18:00:22.310 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:188: TEST_rados_put_get_lrc_kml: delete_pool pool-lrc 2026-02-20T18:00:22.310 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-lrc 2026-02-20T18:00:22.310 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-lrc pool-lrc --yes-i-really-really-mean-it 2026-02-20T18:00:22.585 INFO:tasks.workunit.client.0.trial195.stderr:pool 'pool-lrc' does not exist 2026-02-20T18:00:22.593 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:189: TEST_rados_put_get_lrc_kml: ceph osd erasure-code-profile rm profile-lrc 2026-02-20T18:00:22.869 INFO:tasks.workunit.client.0.trial195.stderr:erasure-code-profile profile-lrc does not exist 2026-02-20T18:00:22.878 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-20T18:00:22.878 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_shec td/test-erasure-code 2026-02-20T18:00:22.879 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:234: TEST_rados_put_get_shec: local dir=td/test-erasure-code 2026-02-20T18:00:22.879 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:236: TEST_rados_put_get_shec: local poolname=pool-shec 2026-02-20T18:00:22.879 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:237: TEST_rados_put_get_shec: local profile=profile-shec 2026-02-20T18:00:22.879 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:239: TEST_rados_put_get_shec: ceph osd erasure-code-profile set profile-shec plugin=shec k=2 m=1 c=1 crush-failure-domain=osd 2026-02-20T18:00:23.163 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:243: TEST_rados_put_get_shec: create_pool pool-shec 12 12 erasure profile-shec 2026-02-20T18:00:23.163 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-shec 12 12 erasure profile-shec 2026-02-20T18:00:23.534 INFO:tasks.workunit.client.0.trial195.stderr:pool 'pool-shec' already exists 2026-02-20T18:00:23.542 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-20T18:00:24.544 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:246: TEST_rados_put_get_shec: rados_put_get td/test-erasure-code pool-shec 2026-02-20T18:00:24.544 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-20T18:00:24.544 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-shec 2026-02-20T18:00:24.544 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-20T18:00:24.544 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-20T18:00:24.545 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-20T18:00:24.545 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-20T18:00:24.545 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-20T18:00:24.545 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-20T18:00:24.545 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-20T18:00:24.545 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-20T18:00:24.545 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-20T18:00:24.545 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-shec put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-20T18:00:24.564 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-shec get SOMETHING td/test-erasure-code/COPY 2026-02-20T18:00:24.583 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-20T18:00:24.584 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-20T18:00:24.585 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-shec SOMETHING 2026-02-20T18:00:24.585 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-shec 2026-02-20T18:00:24.585 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-20T18:00:24.586 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-shec SOMETHING 2026-02-20T18:00:24.586 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:00:24.818 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=7 2026-02-20T18:00:24.818 INFO:tasks.workunit.client.0.trial195.stderr:9 2026-02-20T18:00:24.818 INFO:tasks.workunit.client.0.trial195.stderr:1' 2026-02-20T18:00:24.819 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 7 9 1 2026-02-20T18:00:24.819 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('7' '9' '1') 2026-02-20T18:00:24.819 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-20T18:00:24.819 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=2 2026-02-20T18:00:24.819 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 1 2026-02-20T18:00:25.065 INFO:tasks.workunit.client.0.trial195.stderr:osd.1 is already out. 2026-02-20T18:00:25.073 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-20T18:00:30.075 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-shec SOMETHING 2026-02-20T18:00:30.075 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<1\>' 2026-02-20T18:00:30.075 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-shec 2026-02-20T18:00:30.075 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-20T18:00:30.075 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-shec SOMETHING 2026-02-20T18:00:30.075 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:00:30.304 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=7 2026-02-20T18:00:30.304 INFO:tasks.workunit.client.0.trial195.stderr:9 2026-02-20T18:00:30.304 INFO:tasks.workunit.client.0.trial195.stderr:6' 2026-02-20T18:00:30.304 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 7 9 6 2026-02-20T18:00:30.304 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-shec get SOMETHING td/test-erasure-code/COPY 2026-02-20T18:00:30.323 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-20T18:00:30.324 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 1 2026-02-20T18:00:30.553 INFO:tasks.workunit.client.0.trial195.stderr:osd.1 is already in. 2026-02-20T18:00:30.561 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-20T18:00:30.562 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:248: TEST_rados_put_get_shec: delete_pool pool-shec 2026-02-20T18:00:30.562 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-shec 2026-02-20T18:00:30.562 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-shec pool-shec --yes-i-really-really-mean-it 2026-02-20T18:00:30.855 INFO:tasks.workunit.client.0.trial195.stderr:pool 'pool-shec' does not exist 2026-02-20T18:00:30.864 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:249: TEST_rados_put_get_shec: ceph osd erasure-code-profile rm profile-shec 2026-02-20T18:00:31.140 INFO:tasks.workunit.client.0.trial195.stderr:erasure-code-profile profile-shec does not exist 2026-02-20T18:00:31.148 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:51: run: delete_pool ecpool 2026-02-20T18:00:31.148 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=ecpool 2026-02-20T18:00:31.148 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete ecpool ecpool --yes-i-really-really-mean-it 2026-02-20T18:00:31.419 INFO:tasks.workunit.client.0.trial195.stderr:pool 'ecpool' does not exist 2026-02-20T18:00:31.428 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:52: run: teardown td/test-erasure-code 2026-02-20T18:00:31.428 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code 2026-02-20T18:00:31.428 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T18:00:31.428 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code KILL 2026-02-20T18:00:31.429 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:00:31.429 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:00:31.429 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:00:31.430 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:00:31.430 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:00:31.542 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:00:31.542 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T18:00:31.543 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T18:00:31.543 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T18:00:31.544 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T18:00:31.544 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T18:00:31.544 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T18:00:31.545 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:00:31.545 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T18:00:31.545 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T18:00:31.545 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:00:31.546 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T18:00:31.547 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T18:00:31.547 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code 2026-02-20T18:00:31.603 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T18:00:31.603 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:00:31.603 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.58348 2026-02-20T18:00:31.603 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.58348 2026-02-20T18:00:31.605 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T18:00:31.605 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T18:00:31.605 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2401: main: code=0 2026-02-20T18:00:31.605 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: teardown td/test-erasure-code 0 2026-02-20T18:00:31.605 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code 2026-02-20T18:00:31.606 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-20T18:00:31.606 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code KILL 2026-02-20T18:00:31.606 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:00:31.606 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:00:31.606 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:00:31.606 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:00:31.606 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:00:31.607 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:00:31.607 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T18:00:31.607 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T18:00:31.607 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T18:00:31.608 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T18:00:31.608 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T18:00:31.608 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T18:00:31.609 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:00:31.609 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T18:00:31.610 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T18:00:31.610 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:00:31.610 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T18:00:31.611 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-20T18:00:31.611 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code 2026-02-20T18:00:31.612 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T18:00:31.612 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:00:31.613 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.58348 2026-02-20T18:00:31.613 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.58348 2026-02-20T18:00:31.613 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T18:00:31.613 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T18:00:31.614 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2406: main: return 0 2026-02-20T18:00:31.615 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-20T18:00:31.615 DEBUG:teuthology.orchestra.run.trial195:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-20T18:00:31.683 INFO:tasks.workunit:Running workunit erasure-code/test-erasure-eio.sh... 2026-02-20T18:00:31.683 DEBUG:teuthology.orchestra.run.trial195:workunit test erasure-code/test-erasure-eio.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f7870454149f22574086dd2ff55d050585136531 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh 2026-02-20T18:00:31.745 INFO:tasks.workunit.client.0.trial195.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-20T18:00:31.749 INFO:tasks.workunit.client.0.trial195.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-20T18:00:31.749 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-20T18:00:31.749 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-20T18:00:31.749 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export PYTHONWARNINGS=ignore 2026-02-20T18:00:31.749 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: PYTHONWARNINGS=ignore 2026-02-20T18:00:31.749 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: export CEPH_CONF=/dev/null 2026-02-20T18:00:31.749 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: CEPH_CONF=/dev/null 2026-02-20T18:00:31.749 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2397: main: unset CEPH_ARGS 2026-02-20T18:00:31.749 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: local code 2026-02-20T18:00:31.749 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: run td/test-erasure-eio 2026-02-20T18:00:31.750 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:22: run: local dir=td/test-erasure-eio 2026-02-20T18:00:31.750 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:23: run: shift 2026-02-20T18:00:31.750 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:25: run: export CEPH_MON=127.0.0.1:7112 2026-02-20T18:00:31.750 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:25: run: CEPH_MON=127.0.0.1:7112 2026-02-20T18:00:31.750 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:26: run: export CEPH_ARGS 2026-02-20T18:00:31.750 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:27: run: uuidgen 2026-02-20T18:00:31.752 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:27: run: CEPH_ARGS+='--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none ' 2026-02-20T18:00:31.752 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:28: run: CEPH_ARGS+='--mon-host=127.0.0.1:7112 ' 2026-02-20T18:00:31.752 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:29: run: CEPH_ARGS+='--osd_mclock_override_recovery_settings=true ' 2026-02-20T18:00:31.752 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:31: run: set 2026-02-20T18:00:31.752 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:31: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-20T18:00:31.753 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:31: run: local 'funcs=TEST_ec_backfill_unfound 2026-02-20T18:00:31.753 INFO:tasks.workunit.client.0.trial195.stderr:TEST_ec_object_attr_read_error 2026-02-20T18:00:31.753 INFO:tasks.workunit.client.0.trial195.stderr:TEST_ec_recovery_multiple_errors 2026-02-20T18:00:31.753 INFO:tasks.workunit.client.0.trial195.stderr:TEST_ec_recovery_multiple_objects 2026-02-20T18:00:31.753 INFO:tasks.workunit.client.0.trial195.stderr:TEST_ec_recovery_multiple_objects_eio 2026-02-20T18:00:31.753 INFO:tasks.workunit.client.0.trial195.stderr:TEST_ec_recovery_unfound 2026-02-20T18:00:31.753 INFO:tasks.workunit.client.0.trial195.stderr:TEST_ec_single_recovery_error 2026-02-20T18:00:31.753 INFO:tasks.workunit.client.0.trial195.stderr:TEST_rados_get_bad_size_shard_0 2026-02-20T18:00:31.753 INFO:tasks.workunit.client.0.trial195.stderr:TEST_rados_get_bad_size_shard_1 2026-02-20T18:00:31.753 INFO:tasks.workunit.client.0.trial195.stderr:TEST_rados_get_subread_eio_shard_0 2026-02-20T18:00:31.753 INFO:tasks.workunit.client.0.trial195.stderr:TEST_rados_get_subread_eio_shard_1 2026-02-20T18:00:31.753 INFO:tasks.workunit.client.0.trial195.stderr:TEST_rados_get_subread_missing 2026-02-20T18:00:31.753 INFO:tasks.workunit.client.0.trial195.stderr:TEST_rados_get_with_subreadall_eio_shard_0 2026-02-20T18:00:31.754 INFO:tasks.workunit.client.0.trial195.stderr:TEST_rados_get_with_subreadall_eio_shard_1' 2026-02-20T18:00:31.754 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-20T18:00:31.754 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-20T18:00:31.754 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-20T18:00:31.754 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-20T18:00:31.754 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-20T18:00:31.754 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T18:00:31.754 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-20T18:00:31.754 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:00:31.754 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:00:31.754 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:00:31.754 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:00:31.755 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:00:31.756 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:00:31.756 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T18:00:31.756 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T18:00:31.756 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T18:00:31.756 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T18:00:31.756 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T18:00:31.756 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T18:00:31.757 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:00:31.757 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T18:00:31.757 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T18:00:31.757 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:00:31.757 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T18:00:31.758 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T18:00:31.758 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-20T18:00:31.759 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T18:00:31.759 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:00:31.759 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:00:31.760 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.85808 2026-02-20T18:00:31.760 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T18:00:31.760 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T18:00:31.760 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-20T18:00:31.761 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-20T18:00:31.762 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:00:31.762 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:00:31.762 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.85808 2026-02-20T18:00:31.762 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-20T18:00:31.764 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-20T18:00:31.764 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-20T18:00:31.764 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-20T18:00:31.764 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-20T18:00:31.764 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-20T18:00:31.764 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-20T18:00:31.764 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-20T18:00:31.764 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-20T18:00:31.764 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-20T18:00:31.764 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-20T18:00:31.765 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-20T18:00:31.785 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-20T18:00:31.785 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:00:31.785 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:00:31.785 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:00:31.785 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:00:31.785 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:00:31.785 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:00:31.786 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-20T18:00:31.812 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-20T18:00:31.813 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-20T18:00:31.814 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T18:00:31.814 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T18:00:31.814 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-20T18:00:31.814 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-20T18:00:31.814 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T18:00:31.814 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:00:31.814 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:00:31.814 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:00:31.814 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:00:31.814 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:00:31.814 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:00:31.815 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T18:00:31.815 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.85808/ceph-mon.a.asok config get fsid 2026-02-20T18:00:31.858 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-20T18:00:31.858 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T18:00:31.858 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T18:00:31.858 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-20T18:00:31.859 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-20T18:00:31.859 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T18:00:31.859 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:00:31.859 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:00:31.860 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:00:31.860 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:00:31.860 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:00:31.860 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:00:31.860 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T18:00:31.860 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.85808/ceph-mon.a.asok config get mon_host 2026-02-20T18:00:31.903 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-20T18:00:31.903 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-20T18:00:31.904 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-20T18:00:31.904 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-20T18:00:31.904 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-20T18:00:31.904 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-20T18:00:31.904 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-20T18:00:32.005 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-20T18:00:32.005 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:00:32.005 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:00:32.006 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:00:32.006 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:00:32.006 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:00:32.006 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:00:32.006 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T18:00:32.007 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T18:00:32.023 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-20T18:00:32.024 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-20T18:00:32.120 INFO:tasks.workunit.client.0.trial195.stderr:pool 'rbd' already exists 2026-02-20T18:00:32.126 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-20T18:00:33.127 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-20T18:00:33.127 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:00:33.127 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:00:33.127 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:00:33.127 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:00:33.127 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:00:33.128 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:00:33.128 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-20T18:00:33.128 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-mon.a.asok log flush 2026-02-20T18:00:33.170 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T18:00:33.175 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-20T18:00:33.177 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T18:00:31.803+0000 7fd9c6bced80 0 load: jerasure load: lrc 2026-02-20T18:00:33.178 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_backfill_unfound td/test-erasure-eio 2026-02-20T18:00:33.178 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:531: TEST_ec_backfill_unfound: local dir=td/test-erasure-eio 2026-02-20T18:00:33.178 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:532: TEST_ec_backfill_unfound: local objname=myobject 2026-02-20T18:00:33.178 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:533: TEST_ec_backfill_unfound: local lastobj=300 2026-02-20T18:00:33.178 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:535: TEST_ec_backfill_unfound: local testobj=obj250 2026-02-20T18:00:33.178 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:537: TEST_ec_backfill_unfound: ORIG_ARGS='--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:00:33.178 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:538: TEST_ec_backfill_unfound: CEPH_ARGS+=' --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-20T18:00:33.178 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: setup_osds 5 2026-02-20T18:00:33.178 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=5 2026-02-20T18:00:33.178 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-20T18:00:33.178 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 5 - 1 2026-02-20T18:00:33.178 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 4 2026-02-20T18:00:33.179 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:00:33.179 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-20T18:00:33.179 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:00:33.179 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:00:33.179 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-20T18:00:33.179 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:00:33.179 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-20T18:00:33.179 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-20T18:00:33.179 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:00:33.179 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:00:33.179 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:00:33.179 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-20T18:00:33.179 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-20T18:00:33.179 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:00:33.179 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:00:33.179 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:00:33.179 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:00:33.179 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:00:33.179 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:00:33.180 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:00:33.180 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:00:33.180 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:00:33.180 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:00:33.180 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:00:33.180 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:00:33.180 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:00:33.180 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:00:33.180 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:00:33.180 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:00:33.180 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:00:33.180 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:00:33.180 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:00:33.180 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:00:33.180 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:00:33.180 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:00:33.180 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-20T18:00:33.180 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:00:33.181 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=dbfabdf0-e5a7-4d92-9741-327084c5b831 2026-02-20T18:00:33.181 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 dbfabdf0-e5a7-4d92-9741-327084c5b831' 2026-02-20T18:00:33.181 INFO:tasks.workunit.client.0.trial195.stdout:add osd0 dbfabdf0-e5a7-4d92-9741-327084c5b831 2026-02-20T18:00:33.181 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:00:33.195 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBBoZhpJ7aTCxAA6ON1DMRuBv+J9ml4oBFzIA== 2026-02-20T18:00:33.195 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBBoZhpJ7aTCxAA6ON1DMRuBv+J9ml4oBFzIA=="}' 2026-02-20T18:00:33.195 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new dbfabdf0-e5a7-4d92-9741-327084c5b831 -i td/test-erasure-eio/0/new.json 2026-02-20T18:00:33.303 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:00:33.310 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-20T18:00:33.311 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBBoZhpJ7aTCxAA6ON1DMRuBv+J9ml4oBFzIA== --osd-uuid dbfabdf0-e5a7-4d92-9741-327084c5b831 2026-02-20T18:00:33.331 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:33.329+0000 7fc080fe0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:00:33.333 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:33.331+0000 7fc080fe0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:00:33.334 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:33.332+0000 7fc080fe0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:00:33.334 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:33.332+0000 7fc080fe0900 -1 bdev(0x55d4cca8ac00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-20T18:00:33.334 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:33.332+0000 7fc080fe0900 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-20T18:00:33.789 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-20T18:00:33.789 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:00:33.790 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-20T18:00:33.790 INFO:tasks.workunit.client.0.trial195.stdout:adding osd0 key to auth repository 2026-02-20T18:00:33.790 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:00:33.895 INFO:tasks.workunit.client.0.trial195.stdout:start osd.0 2026-02-20T18:00:33.896 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-20T18:00:33.896 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:00:33.896 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:00:33.896 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:00:33.896 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:00:33.915 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:33.913+0000 7f127996e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:00:33.916 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:33.914+0000 7f127996e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:00:33.917 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:33.915+0000 7f127996e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:00:33.995 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-20T18:00:33.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:00:33.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-20T18:00:33.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:00:33.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:00:33.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:00:33.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:00:33.996 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:00:33.997 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:00:33.997 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T18:00:34.096 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:00:34.107 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:34.105+0000 7f127996e900 -1 Falling back to public interface 2026-02-20T18:00:34.358 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:34.355+0000 7f127996e900 -1 osd.0 0 log_to_monitors true 2026-02-20T18:00:35.098 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:00:35.098 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:00:35.098 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:00:35.099 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:00:35.099 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:00:35.099 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T18:00:35.330 INFO:tasks.workunit.client.0.trial195.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3586760361,v1:127.0.0.1:6803/3586760361] [v2:127.0.0.1:6804/3586760361,v1:127.0.0.1:6805/3586760361] exists,up dbfabdf0-e5a7-4d92-9741-327084c5b831 2026-02-20T18:00:35.335 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:00:35.335 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:00:35.335 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:00:35.336 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:00:35.336 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-20T18:00:35.336 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:00:35.336 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:00:35.336 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-20T18:00:35.336 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:00:35.336 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-20T18:00:35.336 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-20T18:00:35.336 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:00:35.336 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:00:35.336 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:00:35.337 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-20T18:00:35.337 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-20T18:00:35.337 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:00:35.337 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:00:35.337 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:00:35.337 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:00:35.337 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:00:35.337 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:00:35.337 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:00:35.337 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:00:35.337 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:00:35.337 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:00:35.338 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:00:35.338 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:00:35.338 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:00:35.338 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:00:35.338 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:00:35.338 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:00:35.338 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:00:35.338 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:00:35.338 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:00:35.338 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:00:35.338 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:00:35.338 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:00:35.338 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-20T18:00:35.338 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:00:35.339 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=539949be-5074-4e35-b811-57ebf5cfdbe6 2026-02-20T18:00:35.339 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 539949be-5074-4e35-b811-57ebf5cfdbe6' 2026-02-20T18:00:35.339 INFO:tasks.workunit.client.0.trial195.stdout:add osd1 539949be-5074-4e35-b811-57ebf5cfdbe6 2026-02-20T18:00:35.339 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:00:35.348 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBDoZhp33WrFBAAPz606jp0dp94SxWliKCe5g== 2026-02-20T18:00:35.348 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBDoZhp33WrFBAAPz606jp0dp94SxWliKCe5g=="}' 2026-02-20T18:00:35.348 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 539949be-5074-4e35-b811-57ebf5cfdbe6 -i td/test-erasure-eio/1/new.json 2026-02-20T18:00:35.577 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:00:35.584 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-20T18:00:35.585 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBDoZhp33WrFBAAPz606jp0dp94SxWliKCe5g== --osd-uuid 539949be-5074-4e35-b811-57ebf5cfdbe6 2026-02-20T18:00:35.605 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:35.603+0000 7fad96eef900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:00:35.606 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:35.605+0000 7fad96eef900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:00:35.607 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:35.605+0000 7fad96eef900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:00:35.608 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:35.606+0000 7fad96eef900 -1 bdev(0x5599192f6c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-20T18:00:35.608 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:35.606+0000 7fad96eef900 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-20T18:00:35.931 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-20T18:00:35.932 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:00:35.932 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-20T18:00:35.932 INFO:tasks.workunit.client.0.trial195.stdout:adding osd1 key to auth repository 2026-02-20T18:00:35.933 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:00:36.248 INFO:tasks.workunit.client.0.trial195.stdout:start osd.1 2026-02-20T18:00:36.249 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-20T18:00:36.249 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:00:36.249 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:00:36.249 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:00:36.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:00:36.266 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:36.264+0000 7f84ed2fe900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:00:36.268 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:36.266+0000 7f84ed2fe900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:00:36.268 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:36.267+0000 7f84ed2fe900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:00:36.418 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:36.416+0000 7f84ed2fe900 -1 Falling back to public interface 2026-02-20T18:00:36.476 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-20T18:00:36.476 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:00:36.477 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-20T18:00:36.477 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:00:36.477 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:00:36.477 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:00:36.477 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:00:36.477 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:00:36.478 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:00:36.478 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:00:36.705 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:36.703+0000 7f84ed2fe900 -1 osd.1 0 log_to_monitors true 2026-02-20T18:00:36.705 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:00:37.706 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:00:37.707 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:00:37.707 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:00:37.708 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:00:37.708 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:00:37.708 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:00:37.935 INFO:tasks.workunit.client.0.trial195.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1167931132,v1:127.0.0.1:6811/1167931132] [v2:127.0.0.1:6812/1167931132,v1:127.0.0.1:6813/1167931132] exists,up 539949be-5074-4e35-b811-57ebf5cfdbe6 2026-02-20T18:00:37.939 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:00:37.939 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:00:37.939 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:00:37.940 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:00:37.940 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-20T18:00:37.940 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:00:37.940 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:00:37.940 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-20T18:00:37.940 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:00:37.940 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-20T18:00:37.940 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-20T18:00:37.940 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:00:37.940 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:00:37.940 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:00:37.940 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-20T18:00:37.940 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-20T18:00:37.940 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:00:37.940 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:00:37.940 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:00:37.940 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:00:37.941 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:00:37.941 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:00:37.941 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:00:37.941 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:00:37.941 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:00:37.941 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:00:37.941 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:00:37.941 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:00:37.941 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:00:37.941 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:00:37.941 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:00:37.941 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:00:37.941 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:00:37.941 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:00:37.941 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:00:37.941 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:00:37.941 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:00:37.941 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:00:37.942 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-20T18:00:37.942 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:00:37.942 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=6b55b97c-83c5-4cf0-8416-8dd1bfb81f42 2026-02-20T18:00:37.942 INFO:tasks.workunit.client.0.trial195.stdout:add osd2 6b55b97c-83c5-4cf0-8416-8dd1bfb81f42 2026-02-20T18:00:37.942 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 6b55b97c-83c5-4cf0-8416-8dd1bfb81f42' 2026-02-20T18:00:37.942 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:00:37.953 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBFoZhplhvGOBAAG4pZ5gGOc8H8MrRk7SnWyg== 2026-02-20T18:00:37.954 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBFoZhplhvGOBAAG4pZ5gGOc8H8MrRk7SnWyg=="}' 2026-02-20T18:00:37.954 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 6b55b97c-83c5-4cf0-8416-8dd1bfb81f42 -i td/test-erasure-eio/2/new.json 2026-02-20T18:00:38.192 INFO:tasks.workunit.client.0.trial195.stdout:2 2026-02-20T18:00:38.200 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-20T18:00:38.201 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBFoZhplhvGOBAAG4pZ5gGOc8H8MrRk7SnWyg== --osd-uuid 6b55b97c-83c5-4cf0-8416-8dd1bfb81f42 2026-02-20T18:00:38.219 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:38.218+0000 7fe682e70900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:00:38.221 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:38.219+0000 7fe682e70900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:00:38.222 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:38.220+0000 7fe682e70900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:00:38.222 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:38.221+0000 7fe682e70900 -1 bdev(0x55b9df74ec00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-20T18:00:38.222 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:38.221+0000 7fe682e70900 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-20T18:00:38.541 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-20T18:00:38.541 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:00:38.542 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-20T18:00:38.542 INFO:tasks.workunit.client.0.trial195.stdout:adding osd2 key to auth repository 2026-02-20T18:00:38.542 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:00:38.853 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-20T18:00:38.853 INFO:tasks.workunit.client.0.trial195.stdout:start osd.2 2026-02-20T18:00:38.854 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:00:38.854 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:00:38.854 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:00:38.854 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:00:38.871 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:38.869+0000 7fde9ab51900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:00:38.872 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:38.870+0000 7fde9ab51900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:00:38.873 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:38.871+0000 7fde9ab51900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:00:38.995 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:38.993+0000 7fde9ab51900 -1 Falling back to public interface 2026-02-20T18:00:39.084 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-20T18:00:39.084 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:00:39.084 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-20T18:00:39.084 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:00:39.084 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:00:39.084 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:00:39.085 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:00:39.085 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:00:39.085 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:00:39.086 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:00:39.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:00:39.381 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:39.379+0000 7fde9ab51900 -1 osd.2 0 log_to_monitors true 2026-02-20T18:00:40.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:00:40.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:00:40.310 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:00:40.310 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:00:40.310 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:00:40.310 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:00:40.536 INFO:tasks.workunit.client.0.trial195.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3972002596,v1:127.0.0.1:6819/3972002596] [v2:127.0.0.1:6820/3972002596,v1:127.0.0.1:6821/3972002596] exists,up 6b55b97c-83c5-4cf0-8416-8dd1bfb81f42 2026-02-20T18:00:40.540 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:00:40.540 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:00:40.540 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:00:40.540 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:00:40.540 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-20T18:00:40.540 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:00:40.540 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:00:40.540 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-20T18:00:40.541 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:00:40.541 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-20T18:00:40.541 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-20T18:00:40.541 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:00:40.541 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:00:40.541 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:00:40.541 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-20T18:00:40.541 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-20T18:00:40.541 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:00:40.541 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:00:40.541 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:00:40.542 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:00:40.542 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:00:40.542 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:00:40.542 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:00:40.542 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:00:40.542 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:00:40.542 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:00:40.542 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:00:40.542 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:00:40.542 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:00:40.542 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:00:40.542 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:00:40.543 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:00:40.543 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:00:40.543 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:00:40.543 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:00:40.543 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:00:40.543 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:00:40.543 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:00:40.543 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-20T18:00:40.543 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:00:40.543 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=48d2c850-46f9-4a07-87fd-389c5ae35884 2026-02-20T18:00:40.543 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 48d2c850-46f9-4a07-87fd-389c5ae35884' 2026-02-20T18:00:40.544 INFO:tasks.workunit.client.0.trial195.stdout:add osd3 48d2c850-46f9-4a07-87fd-389c5ae35884 2026-02-20T18:00:40.544 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:00:40.553 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBIoZhpSibuIBAA3CsNwja8zOLJSdMu5wR+wQ== 2026-02-20T18:00:40.553 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBIoZhpSibuIBAA3CsNwja8zOLJSdMu5wR+wQ=="}' 2026-02-20T18:00:40.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 48d2c850-46f9-4a07-87fd-389c5ae35884 -i td/test-erasure-eio/3/new.json 2026-02-20T18:00:40.781 INFO:tasks.workunit.client.0.trial195.stdout:3 2026-02-20T18:00:40.788 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-20T18:00:40.789 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBIoZhpSibuIBAA3CsNwja8zOLJSdMu5wR+wQ== --osd-uuid 48d2c850-46f9-4a07-87fd-389c5ae35884 2026-02-20T18:00:40.810 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:40.808+0000 7f02e9791900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:00:40.811 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:40.810+0000 7f02e9791900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:00:40.812 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:40.811+0000 7f02e9791900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:00:40.813 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:40.811+0000 7f02e9791900 -1 bdev(0x55d41e70ac00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-20T18:00:40.813 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:40.811+0000 7f02e9791900 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-20T18:00:41.138 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-20T18:00:41.138 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:00:41.139 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-20T18:00:41.139 INFO:tasks.workunit.client.0.trial195.stdout:adding osd3 key to auth repository 2026-02-20T18:00:41.139 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:00:41.457 INFO:tasks.workunit.client.0.trial195.stdout:start osd.3 2026-02-20T18:00:41.457 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-20T18:00:41.458 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:00:41.458 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:00:41.458 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:00:41.458 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:00:41.476 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:41.474+0000 7f1751812900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:00:41.477 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:41.476+0000 7f1751812900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:00:41.478 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:41.477+0000 7f1751812900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:00:41.601 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:41.599+0000 7f1751812900 -1 Falling back to public interface 2026-02-20T18:00:41.687 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-20T18:00:41.688 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:00:41.688 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-20T18:00:41.688 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:00:41.688 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:00:41.688 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:00:41.688 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:00:41.689 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:00:41.689 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:00:41.689 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T18:00:41.913 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:00:41.998 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:41.996+0000 7f1751812900 -1 osd.3 0 log_to_monitors true 2026-02-20T18:00:42.914 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:00:42.914 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:00:42.914 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:00:42.915 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:00:42.915 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:00:42.915 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T18:00:43.143 INFO:tasks.workunit.client.0.trial195.stdout:osd.3 up in weight 1 up_from 26 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3805509833,v1:127.0.0.1:6827/3805509833] [v2:127.0.0.1:6828/3805509833,v1:127.0.0.1:6829/3805509833] exists,up 48d2c850-46f9-4a07-87fd-389c5ae35884 2026-02-20T18:00:43.147 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:00:43.147 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:00:43.148 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:00:43.148 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:00:43.148 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-20T18:00:43.148 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:00:43.148 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:00:43.148 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-20T18:00:43.148 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:00:43.148 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-20T18:00:43.148 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-20T18:00:43.148 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:00:43.148 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:00:43.149 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:00:43.149 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-20T18:00:43.149 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-20T18:00:43.149 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:00:43.149 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:00:43.149 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:00:43.149 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:00:43.149 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:00:43.149 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:00:43.149 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:00:43.149 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:00:43.150 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:00:43.150 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:00:43.150 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:00:43.150 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:00:43.150 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:00:43.150 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:00:43.150 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:00:43.150 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:00:43.150 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:00:43.150 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:00:43.150 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:00:43.150 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:00:43.150 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:00:43.151 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:00:43.151 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-20T18:00:43.151 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:00:43.151 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c517f598-7dbb-43da-bb63-2a61f22bf602 2026-02-20T18:00:43.151 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 c517f598-7dbb-43da-bb63-2a61f22bf602' 2026-02-20T18:00:43.151 INFO:tasks.workunit.client.0.trial195.stdout:add osd4 c517f598-7dbb-43da-bb63-2a61f22bf602 2026-02-20T18:00:43.152 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:00:43.161 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBLoZhpyQWOCRAA8vuYhuOKT00EreqHDj7k8Q== 2026-02-20T18:00:43.161 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBLoZhpyQWOCRAA8vuYhuOKT00EreqHDj7k8Q=="}' 2026-02-20T18:00:43.161 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c517f598-7dbb-43da-bb63-2a61f22bf602 -i td/test-erasure-eio/4/new.json 2026-02-20T18:00:43.392 INFO:tasks.workunit.client.0.trial195.stdout:4 2026-02-20T18:00:43.401 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-20T18:00:43.402 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBLoZhpyQWOCRAA8vuYhuOKT00EreqHDj7k8Q== --osd-uuid c517f598-7dbb-43da-bb63-2a61f22bf602 2026-02-20T18:00:43.422 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:43.420+0000 7f16d5577900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:00:43.423 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:43.422+0000 7f16d5577900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:00:43.424 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:43.423+0000 7f16d5577900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:00:43.425 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:43.423+0000 7f16d5577900 -1 bdev(0x557a3b002c00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-20T18:00:43.425 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:43.423+0000 7f16d5577900 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-20T18:00:43.768 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-20T18:00:43.769 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:00:43.769 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-20T18:00:43.770 INFO:tasks.workunit.client.0.trial195.stdout:adding osd4 key to auth repository 2026-02-20T18:00:43.770 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:00:44.088 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-20T18:00:44.089 INFO:tasks.workunit.client.0.trial195.stdout:start osd.4 2026-02-20T18:00:44.090 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:00:44.090 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:00:44.090 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:00:44.090 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:00:44.108 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:44.105+0000 7fe816769900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:00:44.109 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:44.107+0000 7fe816769900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:00:44.110 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:44.108+0000 7fe816769900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:00:44.250 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:44.248+0000 7fe816769900 -1 Falling back to public interface 2026-02-20T18:00:44.313 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-20T18:00:44.313 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:00:44.313 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-20T18:00:44.314 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:00:44.314 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:00:44.314 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:00:44.314 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:00:44.315 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:00:44.315 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:00:44.315 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-20T18:00:44.535 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:00:44.615 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:44.613+0000 7fe816769900 -1 osd.4 0 log_to_monitors true 2026-02-20T18:00:45.536 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:00:45.537 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:00:45.537 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:00:45.538 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:00:45.538 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:00:45.538 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-20T18:00:45.764 INFO:tasks.workunit.client.0.trial195.stdout:osd.4 up in weight 1 up_from 35 up_thru 36 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/2897442498,v1:127.0.0.1:6835/2897442498] [v2:127.0.0.1:6836/2897442498,v1:127.0.0.1:6837/2897442498] exists,up c517f598-7dbb-43da-bb63-2a61f22bf602 2026-02-20T18:00:45.765 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:00:45.765 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:00:45.765 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:00:45.766 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-20T18:00:45.766 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-20T18:00:45.766 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-20T18:00:45.766 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:00:45.766 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:00:45.766 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:00:45.766 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.0.asok 2026-02-20T18:00:45.766 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-20T18:00:45.766 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-osd.0.asok log flush 2026-02-20T18:00:45.811 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T18:00:45.815 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-20T18:00:45.817 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T18:00:34.107+0000 7f127996e900 0 load: jerasure load: lrc 2026-02-20T18:00:45.818 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:540: TEST_ec_backfill_unfound: CEPH_ARGS='--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:00:45.818 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:542: TEST_ec_backfill_unfound: local poolname=pool-jerasure 2026-02-20T18:00:45.819 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:543: TEST_ec_backfill_unfound: create_erasure_coded_pool pool-jerasure 3 2 2026-02-20T18:00:45.819 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-20T18:00:45.819 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-20T18:00:45.819 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-20T18:00:45.819 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-20T18:00:45.819 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-20T18:00:45.819 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-20T18:00:45.819 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-20T18:00:46.101 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-20T18:00:46.101 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-20T18:00:46.473 INFO:tasks.workunit.client.0.trial195.stderr:pool 'pool-jerasure' already exists 2026-02-20T18:00:46.482 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-20T18:00:47.483 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-20T18:00:47.484 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:00:47.484 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:00:47.484 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:00:47.485 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:00:47.485 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:00:47.485 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:00:47.485 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:00:47.485 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:00:47.485 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:00:47.568 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:00:47.568 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:00:47.569 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:00:47.569 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:00:47.569 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:00:47.569 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:00:47.791 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:00:47.791 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:00:47.791 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:00:47.791 INFO:tasks.workunit.client.0.trial195.stderr:3 2026-02-20T18:00:47.791 INFO:tasks.workunit.client.0.trial195.stderr:4' 2026-02-20T18:00:47.791 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:00:47.792 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:00:47.792 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:00:47.855 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803781 2026-02-20T18:00:47.856 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803781 2026-02-20T18:00:47.856 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781' 2026-02-20T18:00:47.856 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:00:47.856 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:00:47.920 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607557 2026-02-20T18:00:47.920 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607557 2026-02-20T18:00:47.920 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781 1-51539607557' 2026-02-20T18:00:47.920 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:00:47.920 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:00:47.986 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411332 2026-02-20T18:00:47.986 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411332 2026-02-20T18:00:47.986 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781 1-51539607557 2-77309411332' 2026-02-20T18:00:47.986 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:00:47.986 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:00:48.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149699 2026-02-20T18:00:48.051 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149699 2026-02-20T18:00:48.051 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781 1-51539607557 2-77309411332 3-111669149699' 2026-02-20T18:00:48.051 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:00:48.051 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-20T18:00:48.114 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855363 2026-02-20T18:00:48.115 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855363 2026-02-20T18:00:48.115 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781 1-51539607557 2-77309411332 3-111669149699 4-150323855363' 2026-02-20T18:00:48.115 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:00:48.115 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803781 2026-02-20T18:00:48.115 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:00:48.116 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:00:48.116 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803781 2026-02-20T18:00:48.116 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:00:48.117 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803781 2026-02-20T18:00:48.117 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-20T18:00:48.118 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.0 seq 25769803781 2026-02-20T18:00:48.118 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:00:48.340 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803781 -lt 25769803781 2026-02-20T18:00:48.340 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:00:48.340 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607557 2026-02-20T18:00:48.341 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:00:48.341 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:00:48.342 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-51539607557 2026-02-20T18:00:48.342 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:00:48.343 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=51539607557 2026-02-20T18:00:48.344 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 51539607557' 2026-02-20T18:00:48.344 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.1 seq 51539607557 2026-02-20T18:00:48.344 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:00:48.566 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 51539607557 -lt 51539607557 2026-02-20T18:00:48.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:00:48.567 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411332 2026-02-20T18:00:48.567 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:00:48.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:00:48.568 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-77309411332 2026-02-20T18:00:48.568 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:00:48.569 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=77309411332 2026-02-20T18:00:48.569 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 77309411332' 2026-02-20T18:00:48.570 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.2 seq 77309411332 2026-02-20T18:00:48.570 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:00:48.791 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411332 -lt 77309411332 2026-02-20T18:00:48.791 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:00:48.791 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149699 2026-02-20T18:00:48.791 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:00:48.792 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:00:48.793 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149699 2026-02-20T18:00:48.793 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:00:48.794 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149699 2026-02-20T18:00:48.794 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149699' 2026-02-20T18:00:48.794 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.3 seq 111669149699 2026-02-20T18:00:48.795 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:00:49.019 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149697 -lt 111669149699 2026-02-20T18:00:49.019 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:00:50.020 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:00:50.021 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:00:50.247 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149700 -lt 111669149699 2026-02-20T18:00:50.247 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:00:50.247 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-150323855363 2026-02-20T18:00:50.248 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:00:50.248 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-20T18:00:50.249 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-150323855363 2026-02-20T18:00:50.249 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:00:50.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=150323855363 2026-02-20T18:00:50.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 150323855363' 2026-02-20T18:00:50.250 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.4 seq 150323855363 2026-02-20T18:00:50.251 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-20T18:00:50.473 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 150323855363 -lt 150323855363 2026-02-20T18:00:50.474 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:00:50.474 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:00:50.474 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:00:50.787 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:00:50.787 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:00:50.787 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:00:50.787 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:00:50.787 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:00:50.788 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:00:50.788 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:00:50.788 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:00:51.007 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:00:51.008 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:00:51.009 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:00:51.009 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:00:51.324 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:00:51.324 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:00:51.324 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:00:51.324 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:545: TEST_ec_backfill_unfound: ceph pg dump pgs 2026-02-20T18:00:51.539 INFO:tasks.workunit.client.0.trial195.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-20T18:00:51.539 INFO:tasks.workunit.client.0.trial195.stdout:2.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-20T18:00:47.300633+0000 0'0 42:13 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-20T18:00:46.410154+0000 0'0 2026-02-20T18:00:46.410154+0000 0 0 periodic scrub scheduled @ 2026-02-21T19:41:56.330201+0000 0 0 2026-02-20T18:00:51.539 INFO:tasks.workunit.client.0.trial195.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-20T18:00:45.717339+0000 0'0 42:22 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-20T18:00:32.101796+0000 0'0 2026-02-20T18:00:32.101796+0000 0 0 periodic scrub scheduled @ 2026-02-21T18:03:17.864043+0000 0 0 2026-02-20T18:00:51.539 INFO:tasks.workunit.client.0.trial195.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-20T18:00:45.717422+0000 0'0 42:102 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-20T18:00:32.101796+0000 0'0 2026-02-20T18:00:32.101796+0000 0 0 periodic scrub scheduled @ 2026-02-22T04:06:21.541002+0000 0 0 2026-02-20T18:00:51.539 INFO:tasks.workunit.client.0.trial195.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-20T18:00:45.717045+0000 0'0 42:22 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-20T18:00:32.101796+0000 0'0 2026-02-20T18:00:32.101796+0000 0 0 periodic scrub scheduled @ 2026-02-22T02:51:06.758508+0000 0 0 2026-02-20T18:00:51.539 INFO:tasks.workunit.client.0.trial195.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-20T18:00:40.820367+0000 0'0 43:67 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-20T18:00:32.101796+0000 0'0 2026-02-20T18:00:32.101796+0000 0 0 periodic scrub scheduled @ 2026-02-21T20:19:24.364591+0000 0 0 2026-02-20T18:00:51.539 INFO:tasks.workunit.client.0.trial195.stdout: 2026-02-20T18:00:51.539 INFO:tasks.workunit.client.0.trial195.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-20T18:00:51.540 INFO:tasks.workunit.client.0.trial195.stderr:dumped pgs 2026-02-20T18:00:51.547 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:547: TEST_ec_backfill_unfound: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-20T18:00:51.547 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-20T18:00:51.547 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-20T18:00:51.547 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-20T18:00:51.547 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:00:51.547 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-20T18:00:51.547 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:00:51.547 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-20T18:00:51.547 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:00:51.547 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-20T18:00:51.548 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:00:51.548 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-20T18:00:51.548 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:51.568 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:548: TEST_ec_backfill_unfound: get_primary pool-jerasure myobject 2026-02-20T18:00:51.568 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=pool-jerasure 2026-02-20T18:00:51.568 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=myobject 2026-02-20T18:00:51.568 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map pool-jerasure myobject 2026-02-20T18:00:51.568 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-20T18:00:51.790 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:548: TEST_ec_backfill_unfound: local primary=3 2026-02-20T18:00:51.791 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:550: TEST_ec_backfill_unfound: get_osds pool-jerasure myobject 2026-02-20T18:00:51.791 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:00:51.791 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-20T18:00:51.792 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-20T18:00:51.792 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:00:52.020 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T18:00:52.020 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:00:52.020 INFO:tasks.workunit.client.0.trial195.stderr:4 2026-02-20T18:00:52.020 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T18:00:52.020 INFO:tasks.workunit.client.0.trial195.stderr:2' 2026-02-20T18:00:52.020 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2 2026-02-20T18:00:52.021 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:550: TEST_ec_backfill_unfound: initial_osds=('3' '1' '4' '0' '2') 2026-02-20T18:00:52.021 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:550: TEST_ec_backfill_unfound: local -a initial_osds 2026-02-20T18:00:52.021 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:551: TEST_ec_backfill_unfound: local last_osd=2 2026-02-20T18:00:52.021 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:552: TEST_ec_backfill_unfound: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-20T18:00:52.022 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:00:52.022 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:00:52.022 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:00:52.022 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:00:52.022 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:00:52.126 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:00:52.127 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:553: TEST_ec_backfill_unfound: ceph osd down 2 2026-02-20T18:00:52.351 INFO:tasks.workunit.client.0.trial195.stderr:osd.2 is already down. 2026-02-20T18:00:52.358 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:554: TEST_ec_backfill_unfound: ceph osd out 2 2026-02-20T18:00:52.635 INFO:tasks.workunit.client.0.trial195.stderr:osd.2 is already out. 2026-02-20T18:00:52.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:556: TEST_ec_backfill_unfound: ceph pg dump pgs 2026-02-20T18:00:52.856 INFO:tasks.workunit.client.0.trial195.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-20T18:00:52.857 INFO:tasks.workunit.client.0.trial195.stdout:2.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-20T18:00:47.300633+0000 0'0 42:13 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-20T18:00:46.410154+0000 0'0 2026-02-20T18:00:46.410154+0000 0 0 periodic scrub scheduled @ 2026-02-21T19:41:56.330201+0000 0 0 2026-02-20T18:00:52.857 INFO:tasks.workunit.client.0.trial195.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-20T18:00:45.717339+0000 0'0 43:24 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-20T18:00:32.101796+0000 0'0 2026-02-20T18:00:32.101796+0000 0 0 periodic scrub scheduled @ 2026-02-21T18:03:17.864043+0000 0 0 2026-02-20T18:00:52.857 INFO:tasks.workunit.client.0.trial195.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-20T18:00:45.717422+0000 0'0 43:104 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-20T18:00:32.101796+0000 0'0 2026-02-20T18:00:32.101796+0000 0 0 periodic scrub scheduled @ 2026-02-22T04:06:21.541002+0000 0 0 2026-02-20T18:00:52.857 INFO:tasks.workunit.client.0.trial195.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-20T18:00:45.717045+0000 0'0 43:24 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-20T18:00:32.101796+0000 0'0 2026-02-20T18:00:32.101796+0000 0 0 periodic scrub scheduled @ 2026-02-22T02:51:06.758508+0000 0 0 2026-02-20T18:00:52.857 INFO:tasks.workunit.client.0.trial195.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-20T18:00:40.820367+0000 0'0 43:67 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-20T18:00:32.101796+0000 0'0 2026-02-20T18:00:32.101796+0000 0 0 periodic scrub scheduled @ 2026-02-21T20:19:24.364591+0000 0 0 2026-02-20T18:00:52.857 INFO:tasks.workunit.client.0.trial195.stdout: 2026-02-20T18:00:52.857 INFO:tasks.workunit.client.0.trial195.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-20T18:00:52.858 INFO:tasks.workunit.client.0.trial195.stderr:dumped pgs 2026-02-20T18:00:52.864 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:558: TEST_ec_backfill_unfound: dd if=/dev/urandom of=td/test-erasure-eio/ORIGINAL bs=1024 count=4 2026-02-20T18:00:52.865 INFO:tasks.workunit.client.0.trial195.stderr:4+0 records in 2026-02-20T18:00:52.865 INFO:tasks.workunit.client.0.trial195.stderr:4+0 records out 2026-02-20T18:00:52.865 INFO:tasks.workunit.client.0.trial195.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 6.9229e-05 s, 59.2 MB/s 2026-02-20T18:00:52.865 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: seq 1 300 2026-02-20T18:00:52.866 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:52.866 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj1 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:52.887 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:52.887 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj2 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:52.907 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:52.907 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj3 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:52.926 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:52.926 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj4 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:52.945 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:52.945 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj5 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:52.964 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:52.964 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj6 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:52.983 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:52.983 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj7 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.002 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.002 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj8 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.021 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.021 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj9 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.040 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.040 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj10 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.059 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.059 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj11 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj12 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.098 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.098 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj13 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.118 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.118 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj14 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.136 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.136 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj15 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.155 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.156 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj16 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.175 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.175 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj17 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.194 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.194 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj18 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.213 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.213 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj19 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.232 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.232 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj20 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.251 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.251 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj21 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.270 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.270 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj22 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.288 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.288 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj23 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj24 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.326 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.326 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj25 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.343 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.343 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj26 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.363 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.363 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj27 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.382 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.382 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj28 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.401 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.401 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj29 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.420 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.420 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj30 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.439 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.439 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj31 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.457 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.457 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj32 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.477 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.477 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj33 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.495 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.495 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj34 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.514 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.514 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj35 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.533 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.533 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj36 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.551 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.552 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj37 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.570 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.570 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj38 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.589 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.589 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj39 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.608 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.608 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj40 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.627 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.627 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj41 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.645 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.645 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj42 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.664 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.664 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj43 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.684 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.684 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj44 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.703 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.703 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj45 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.722 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.722 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj46 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.741 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.741 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj47 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.759 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.759 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj48 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.778 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.778 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj49 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.796 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.796 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj50 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.815 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.815 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj51 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.834 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.835 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj52 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.853 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.854 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj53 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.872 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.872 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj54 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.891 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.891 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj55 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.910 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.910 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj56 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.928 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.929 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj57 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.947 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.948 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj58 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.966 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.966 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj59 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:53.986 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:53.986 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj60 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.005 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.005 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj61 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.025 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.025 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj62 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.045 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.045 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj63 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.064 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.064 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj64 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.084 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.084 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj65 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.104 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.104 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj66 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.123 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.123 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj67 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.141 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.141 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj68 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.160 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.161 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj69 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.179 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.180 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj70 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.198 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.198 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj71 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.217 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.217 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj72 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj73 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.255 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.255 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj74 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.274 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.274 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj75 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.293 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.293 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj76 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.312 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.312 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj77 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.331 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.331 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj78 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.350 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.350 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj79 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.369 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.369 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj80 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.388 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.388 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj81 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.407 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.407 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj82 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.426 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.427 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj83 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.446 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.446 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj84 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.465 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.466 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj85 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.484 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.484 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj86 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.503 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.504 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj87 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.522 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.522 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj88 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.540 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.541 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj89 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.559 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.559 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj90 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.578 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.578 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj91 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.598 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.598 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj92 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.616 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.616 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj93 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.636 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.636 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj94 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.654 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.655 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj95 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.673 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.673 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj96 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.692 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.692 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj97 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.710 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.710 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj98 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.730 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.730 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj99 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.748 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.749 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj100 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.766 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.766 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj101 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.785 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.786 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj102 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.805 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.805 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj103 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.824 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.824 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj104 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.842 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.843 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj105 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.860 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.860 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj106 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.879 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.879 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj107 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.898 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.898 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj108 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.917 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.918 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj109 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.937 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.937 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj110 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.956 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.956 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj111 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.976 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj112 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:54.994 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:54.994 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj113 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.013 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.013 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj114 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.031 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.032 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj115 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.052 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.052 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj116 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.071 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.071 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj117 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.090 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.090 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj118 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.110 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.110 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj119 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.128 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.129 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj120 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.149 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.149 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj121 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.168 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.168 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj122 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.187 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.187 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj123 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.206 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.206 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj124 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.225 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.225 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj125 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.245 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.245 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj126 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.263 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.263 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj127 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.282 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.282 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj128 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.301 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.301 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj129 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.320 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.320 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj130 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.340 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.340 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj131 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.359 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.359 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj132 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.378 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.379 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj133 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.397 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.397 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj134 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.417 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.417 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj135 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.435 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.436 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj136 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.455 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.455 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj137 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.474 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.475 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj138 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.493 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.493 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj139 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.512 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.512 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj140 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.530 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.530 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj141 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.549 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.550 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj142 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.568 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.568 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj143 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.586 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.587 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj144 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.605 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.605 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj145 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.623 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.624 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj146 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.642 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.642 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj147 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.661 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.661 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj148 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.679 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.679 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj149 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.698 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.698 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj150 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.718 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.718 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj151 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.739 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.739 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj152 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.758 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.758 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj153 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.776 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.776 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj154 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.795 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.795 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj155 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.814 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.814 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj156 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.834 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.834 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj157 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.853 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.853 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj158 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.872 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.872 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj159 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.891 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.891 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj160 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.910 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.910 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj161 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.930 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.930 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj162 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.949 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.949 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj163 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.968 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.969 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj164 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:55.988 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:55.988 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj165 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.007 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.007 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj166 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.027 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.027 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj167 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.046 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.046 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj168 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.064 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.064 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj169 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.083 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.083 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj170 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.102 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.102 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj171 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.122 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.122 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj172 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.141 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.142 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj173 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.161 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.161 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj174 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.180 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.180 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj175 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.199 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.199 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj176 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.218 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.219 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj177 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj178 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.257 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.257 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj179 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.276 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.276 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj180 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.295 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.295 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj181 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.314 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.314 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj182 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.334 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.334 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj183 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.353 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.353 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj184 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.372 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.372 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj185 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.391 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.391 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj186 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.411 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.411 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj187 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.430 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.430 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj188 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.449 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.449 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj189 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.468 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.469 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj190 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.487 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.487 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj191 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.506 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.506 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj192 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.526 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.526 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj193 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.545 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.545 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj194 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.564 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.564 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj195 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.582 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.583 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj196 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.601 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.602 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj197 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.621 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj198 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.639 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.639 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj199 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.657 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.657 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj200 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.676 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.676 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj201 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.694 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.695 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj202 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.714 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.714 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj203 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.733 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.733 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj204 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.752 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.752 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj205 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.770 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.770 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj206 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.788 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.788 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj207 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.807 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.808 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj208 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.827 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.827 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj209 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.846 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.846 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj210 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.866 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.866 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj211 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.885 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.886 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj212 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.906 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.906 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj213 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.925 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.925 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj214 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj215 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.962 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.962 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj216 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:56.982 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:56.982 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj217 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.001 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.001 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj218 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.020 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.020 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj219 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.040 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.040 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj220 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.059 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.059 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj221 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj222 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.097 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.098 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj223 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.117 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.117 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj224 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.136 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.136 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj225 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.157 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.157 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj226 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.176 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.176 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj227 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.194 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.194 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj228 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.213 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.214 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj229 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.232 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.233 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj230 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj231 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.271 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.271 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj232 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.290 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.290 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj233 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj234 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.328 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.328 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj235 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.347 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.347 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj236 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.366 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.366 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj237 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.384 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.384 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj238 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.403 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.403 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj239 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.422 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.422 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj240 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.441 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.441 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj241 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.459 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.459 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj242 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.478 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.478 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj243 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.497 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.498 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj244 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.518 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.518 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj245 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.537 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.538 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj246 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.556 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.556 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj247 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.575 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.575 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj248 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.593 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.593 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj249 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.612 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.612 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj250 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.630 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.630 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj251 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.649 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.649 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj252 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj253 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.687 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.687 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj254 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.706 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.706 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj255 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.724 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.724 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj256 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.745 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.745 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj257 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.763 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.763 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj258 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.782 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.782 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj259 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.801 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.801 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj260 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.820 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.820 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj261 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.839 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.839 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj262 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.857 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.857 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj263 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.875 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.875 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj264 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.894 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.894 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj265 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.912 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.912 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj266 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.931 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.931 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj267 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.950 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.950 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj268 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.969 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.969 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj269 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:57.989 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:57.989 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj270 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:58.007 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:58.007 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj271 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:58.026 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:58.026 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj272 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:58.045 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:58.045 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj273 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:58.065 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:58.065 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj274 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:58.084 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:58.084 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj275 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:58.103 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:58.103 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj276 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:58.122 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:58.123 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj277 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:58.141 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:58.141 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj278 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:58.159 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:58.159 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj279 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:58.178 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:58.178 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj280 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:58.197 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:58.197 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj281 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:58.216 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:58.216 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj282 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:58.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:58.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj283 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:58.253 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:58.253 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj284 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:58.272 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:58.272 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj285 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:58.292 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:58.292 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj286 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:58.311 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:58.311 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj287 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:58.330 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:58.330 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj288 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:58.350 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:58.350 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj289 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:58.369 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:58.369 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj290 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:58.388 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:58.388 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj291 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:58.406 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:58.406 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj292 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:58.426 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:58.426 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj293 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:58.444 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:58.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj294 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:58.463 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:58.464 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj295 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:58.481 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:58.481 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj296 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:58.500 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:58.501 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj297 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:58.519 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:58.519 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj298 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:58.538 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:58.539 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj299 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:58.558 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:00:58.558 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj300 td/test-erasure-eio/ORIGINAL 2026-02-20T18:00:58.578 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:564: TEST_ec_backfill_unfound: inject_eio ec data pool-jerasure obj250 td/test-erasure-eio 0 2026-02-20T18:00:58.578 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=ec 2026-02-20T18:00:58.578 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-20T18:00:58.578 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=data 2026-02-20T18:00:58.578 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-20T18:00:58.578 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=pool-jerasure 2026-02-20T18:00:58.578 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-20T18:00:58.579 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=obj250 2026-02-20T18:00:58.579 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-20T18:00:58.579 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/test-erasure-eio 2026-02-20T18:00:58.579 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-20T18:00:58.579 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=0 2026-02-20T18:00:58.579 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-20T18:00:58.579 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds pool-jerasure obj250 2026-02-20T18:00:58.579 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:00:58.579 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj250 2026-02-20T18:00:58.579 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj250 2026-02-20T18:00:58.579 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:00:58.812 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T18:00:58.812 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:00:58.812 INFO:tasks.workunit.client.0.trial195.stderr:4 2026-02-20T18:00:58.812 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T18:00:58.813 INFO:tasks.workunit.client.0.trial195.stderr:2147483647' 2026-02-20T18:00:58.813 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2147483647 2026-02-20T18:00:58.813 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('3' '1' '4' '0' '2147483647') 2026-02-20T18:00:58.813 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-20T18:00:58.813 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=3 2026-02-20T18:00:58.813 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' ec '!=' ec ']' 2026-02-20T18:00:58.813 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/test-erasure-eio/3/type 2026-02-20T18:00:58.815 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-20T18:00:58.815 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-20T18:00:58.815 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-20T18:00:58.816 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-20T18:00:58.816 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-20T18:00:58.816 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-20T18:00:58.816 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-20T18:00:58.816 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-20T18:00:58.816 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-20T18:00:58.816 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-20T18:00:58.816 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:00:58.816 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:00:58.816 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:00:58.817 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.3.asok 2026-02-20T18:00:58.817 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.85808/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-20T18:00:58.863 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-20T18:00:58.863 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-20T18:00:58.865 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-20T18:00:58.865 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.3 2026-02-20T18:00:58.865 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-20T18:00:58.865 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-20T18:00:58.865 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:00:58.865 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:00:58.865 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:00:58.865 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.3.asok 2026-02-20T18:00:58.865 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-20T18:00:58.865 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-osd.3.asok injectdataerr pool-jerasure obj250 0 2026-02-20T18:00:58.914 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:565: TEST_ec_backfill_unfound: inject_eio ec data pool-jerasure obj250 td/test-erasure-eio 1 2026-02-20T18:00:58.914 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=ec 2026-02-20T18:00:58.914 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-20T18:00:58.914 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=data 2026-02-20T18:00:58.914 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-20T18:00:58.914 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=pool-jerasure 2026-02-20T18:00:58.914 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-20T18:00:58.914 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=obj250 2026-02-20T18:00:58.914 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-20T18:00:58.914 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/test-erasure-eio 2026-02-20T18:00:58.914 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-20T18:00:58.915 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=1 2026-02-20T18:00:58.915 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-20T18:00:58.915 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds pool-jerasure obj250 2026-02-20T18:00:58.915 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:00:58.915 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj250 2026-02-20T18:00:58.915 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj250 2026-02-20T18:00:58.915 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:00:59.146 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T18:00:59.147 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:00:59.147 INFO:tasks.workunit.client.0.trial195.stderr:4 2026-02-20T18:00:59.147 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T18:00:59.147 INFO:tasks.workunit.client.0.trial195.stderr:2147483647' 2026-02-20T18:00:59.147 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2147483647 2026-02-20T18:00:59.147 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('3' '1' '4' '0' '2147483647') 2026-02-20T18:00:59.147 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-20T18:00:59.147 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=1 2026-02-20T18:00:59.147 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' ec '!=' ec ']' 2026-02-20T18:00:59.148 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/test-erasure-eio/1/type 2026-02-20T18:00:59.149 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-20T18:00:59.149 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-20T18:00:59.150 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-20T18:00:59.150 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-20T18:00:59.150 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-20T18:00:59.150 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-20T18:00:59.150 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-20T18:00:59.150 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-20T18:00:59.150 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-20T18:00:59.150 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-20T18:00:59.150 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:00:59.150 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:00:59.150 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:00:59.150 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.1.asok 2026-02-20T18:00:59.151 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.85808/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-20T18:00:59.198 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-20T18:00:59.198 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-20T18:00:59.199 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-20T18:00:59.199 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.1 2026-02-20T18:00:59.200 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-20T18:00:59.200 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-20T18:00:59.200 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:00:59.200 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:00:59.200 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:00:59.200 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.1.asok 2026-02-20T18:00:59.200 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-20T18:00:59.200 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-osd.1.asok injectdataerr pool-jerasure obj250 1 2026-02-20T18:00:59.251 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:567: TEST_ec_backfill_unfound: activate_osd td/test-erasure-eio 2 2026-02-20T18:00:59.251 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-20T18:00:59.251 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-20T18:00:59.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-20T18:00:59.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-20T18:00:59.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-20T18:00:59.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:00:59.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:00:59.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:00:59.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:00:59.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-20T18:00:59.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-20T18:00:59.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-20T18:00:59.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-20T18:00:59.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:00:59.253 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-20T18:00:59.253 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:00:59.253 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:00:59.253 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:00:59.253 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:00:59.253 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:00:59.253 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:00:59.253 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:00:59.253 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:00:59.253 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:00:59.253 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:00:59.253 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:00:59.253 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:00:59.254 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:00:59.254 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:00:59.254 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-20T18:00:59.254 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-20T18:00:59.254 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-20T18:00:59.254 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-20T18:00:59.254 INFO:tasks.workunit.client.0.trial195.stdout:start osd.2 2026-02-20T18:00:59.255 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:00:59.255 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-20T18:00:59.255 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-20T18:00:59.255 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-20T18:00:59.256 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-20T18:00:59.256 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-20T18:00:59.268 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:59.267+0000 7f3985b57900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:00:59.270 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:59.269+0000 7f3985b57900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:00:59.271 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:59.270+0000 7f3985b57900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:00:59.388 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:59.387+0000 7f3985b57900 -1 Falling back to public interface 2026-02-20T18:00:59.483 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-20T18:00:59.483 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:00:59.484 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-20T18:00:59.484 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:00:59.484 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:00:59.484 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:00:59.484 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:00:59.484 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:00:59.485 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:00:59.485 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:00:59.710 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:00:59.733 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:00:59.732+0000 7f3985b57900 -1 osd.2 43 log_to_monitors true 2026-02-20T18:01:00.712 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:01:00.712 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:01:00.712 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:01:00.712 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:01:00.713 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:01:00.713 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:01:00.937 INFO:tasks.workunit.client.0.trial195.stdout:osd.2 up out weight 0 up_from 49 up_thru 20 down_at 44 last_clean_interval [18,43) [v2:127.0.0.1:6818/343353564,v1:127.0.0.1:6819/343353564] [v2:127.0.0.1:6820/343353564,v1:127.0.0.1:6821/343353564] exists,up 6b55b97c-83c5-4cf0-8416-8dd1bfb81f42 2026-02-20T18:01:00.938 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:01:00.938 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:01:00.938 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:01:00.938 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:568: TEST_ec_backfill_unfound: ceph osd in 2 2026-02-20T18:01:01.193 INFO:tasks.workunit.client.0.trial195.stderr:osd.2 is already in. 2026-02-20T18:01:01.201 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:570: TEST_ec_backfill_unfound: sleep 15 2026-02-20T18:01:16.203 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:572: TEST_ec_backfill_unfound: seq 1 240 2026-02-20T18:01:16.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:572: TEST_ec_backfill_unfound: for tmp in $(seq 1 240) 2026-02-20T18:01:16.205 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:573: TEST_ec_backfill_unfound: get_state 2.0 2026-02-20T18:01:16.205 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:60: get_state: local pgid=2.0 2026-02-20T18:01:16.205 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:61: get_state: local sname=state 2026-02-20T18:01:16.206 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:62: get_state: ceph --format json pg dump pgs 2026-02-20T18:01:16.206 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:63: get_state: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .state' 2026-02-20T18:01:16.428 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:573: TEST_ec_backfill_unfound: state=active+backfill_unfound+undersized+degraded+remapped 2026-02-20T18:01:16.429 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:574: TEST_ec_backfill_unfound: echo active+backfill_unfound+undersized+degraded+remapped 2026-02-20T18:01:16.429 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:574: TEST_ec_backfill_unfound: grep backfill_unfound 2026-02-20T18:01:16.430 INFO:tasks.workunit.client.0.trial195.stdout:active+backfill_unfound+undersized+degraded+remapped 2026-02-20T18:01:16.430 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:575: TEST_ec_backfill_unfound: '[' 0 = 0 ']' 2026-02-20T18:01:16.430 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:576: TEST_ec_backfill_unfound: break 2026-02-20T18:01:16.431 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:582: TEST_ec_backfill_unfound: ceph pg dump pgs 2026-02-20T18:01:16.640 INFO:tasks.workunit.client.0.trial195.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-20T18:01:16.640 INFO:tasks.workunit.client.0.trial195.stdout:2.0 301 1 28 0 1 1232896 0 0 101 200 101 active+backfill_unfound+undersized+degraded+remapped 2026-02-20T18:01:01.981332+0000 47'301 56:909 [3,1,4,0,2] 3 [3,1,4,0,NONE] 3 0'0 2026-02-20T18:00:46.410154+0000 0'0 2026-02-20T18:00:46.410154+0000 0 0 no scrub is scheduled 0 0 2026-02-20T18:01:16.640 INFO:tasks.workunit.client.0.trial195.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-20T18:01:01.232388+0000 0'0 56:68 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-20T18:00:32.101796+0000 0'0 2026-02-20T18:00:32.101796+0000 0 0 periodic scrub scheduled @ 2026-02-21T23:05:34.978469+0000 0 0 2026-02-20T18:01:16.640 INFO:tasks.workunit.client.0.trial195.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-20T18:00:45.717422+0000 0'0 56:129 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-20T18:00:32.101796+0000 0'0 2026-02-20T18:00:32.101796+0000 0 0 periodic scrub scheduled @ 2026-02-22T04:06:21.541002+0000 0 0 2026-02-20T18:01:16.640 INFO:tasks.workunit.client.0.trial195.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-20T18:00:45.717045+0000 0'0 56:50 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-20T18:00:32.101796+0000 0'0 2026-02-20T18:00:32.101796+0000 0 0 periodic scrub scheduled @ 2026-02-22T02:51:06.758508+0000 0 0 2026-02-20T18:01:16.641 INFO:tasks.workunit.client.0.trial195.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-20T18:01:01.232363+0000 0'0 56:109 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-20T18:00:32.101796+0000 0'0 2026-02-20T18:00:32.101796+0000 0 0 periodic scrub scheduled @ 2026-02-22T04:44:49.468014+0000 0 0 2026-02-20T18:01:16.641 INFO:tasks.workunit.client.0.trial195.stdout: 2026-02-20T18:01:16.641 INFO:tasks.workunit.client.0.trial195.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-20T18:01:16.641 INFO:tasks.workunit.client.0.trial195.stderr:dumped pgs 2026-02-20T18:01:16.647 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:583: TEST_ec_backfill_unfound: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-20T18:01:16.647 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:01:16.648 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:01:16.648 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:01:16.648 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:01:16.648 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:01:16.753 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:01:16.753 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:584: TEST_ec_backfill_unfound: sleep 5 2026-02-20T18:01:21.754 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:586: TEST_ec_backfill_unfound: ceph pg dump pgs 2026-02-20T18:01:21.970 INFO:tasks.workunit.client.0.trial195.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-20T18:01:21.970 INFO:tasks.workunit.client.0.trial195.stdout:2.0 301 1 303 0 1 1232896 0 0 101 200 101 active+recovery_unfound+undersized+degraded 2026-02-20T18:01:16.808051+0000 47'301 58:921 [3,1,4,0,NONE] 3 [3,1,4,0,NONE] 3 0'0 2026-02-20T18:00:46.410154+0000 0'0 2026-02-20T18:00:46.410154+0000 0 0 no scrub is scheduled 0 0 2026-02-20T18:01:21.971 INFO:tasks.workunit.client.0.trial195.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-20T18:01:01.232388+0000 0'0 56:68 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-20T18:00:32.101796+0000 0'0 2026-02-20T18:00:32.101796+0000 0 0 periodic scrub scheduled @ 2026-02-21T23:05:34.978469+0000 0 0 2026-02-20T18:01:21.971 INFO:tasks.workunit.client.0.trial195.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-20T18:00:45.717422+0000 0'0 56:129 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-20T18:00:32.101796+0000 0'0 2026-02-20T18:00:32.101796+0000 0 0 periodic scrub scheduled @ 2026-02-22T04:06:21.541002+0000 0 0 2026-02-20T18:01:21.971 INFO:tasks.workunit.client.0.trial195.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-20T18:00:45.717045+0000 0'0 56:50 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-20T18:00:32.101796+0000 0'0 2026-02-20T18:00:32.101796+0000 0 0 periodic scrub scheduled @ 2026-02-22T02:51:06.758508+0000 0 0 2026-02-20T18:01:21.971 INFO:tasks.workunit.client.0.trial195.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+undersized 2026-02-20T18:01:16.805788+0000 0'0 58:117 [1,0] 1 [1,0] 1 0'0 2026-02-20T18:00:32.101796+0000 0'0 2026-02-20T18:00:32.101796+0000 0 0 periodic scrub scheduled @ 2026-02-22T03:26:25.370836+0000 0 0 2026-02-20T18:01:21.971 INFO:tasks.workunit.client.0.trial195.stdout: 2026-02-20T18:01:21.971 INFO:tasks.workunit.client.0.trial195.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-20T18:01:21.971 INFO:tasks.workunit.client.0.trial195.stderr:dumped pgs 2026-02-20T18:01:21.977 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:587: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-20T18:01:22.041 INFO:tasks.workunit.client.0.trial195.stdout:{ 2026-02-20T18:01:22.041 INFO:tasks.workunit.client.0.trial195.stdout: "num_missing": 1, 2026-02-20T18:01:22.041 INFO:tasks.workunit.client.0.trial195.stdout: "num_unfound": 1, 2026-02-20T18:01:22.041 INFO:tasks.workunit.client.0.trial195.stdout: "objects": [ 2026-02-20T18:01:22.041 INFO:tasks.workunit.client.0.trial195.stdout: { 2026-02-20T18:01:22.041 INFO:tasks.workunit.client.0.trial195.stdout: "oid": { 2026-02-20T18:01:22.041 INFO:tasks.workunit.client.0.trial195.stdout: "oid": "obj250", 2026-02-20T18:01:22.041 INFO:tasks.workunit.client.0.trial195.stdout: "key": "", 2026-02-20T18:01:22.042 INFO:tasks.workunit.client.0.trial195.stdout: "snapid": -2, 2026-02-20T18:01:22.042 INFO:tasks.workunit.client.0.trial195.stdout: "hash": 2249616407, 2026-02-20T18:01:22.042 INFO:tasks.workunit.client.0.trial195.stdout: "max": 0, 2026-02-20T18:01:22.042 INFO:tasks.workunit.client.0.trial195.stdout: "pool": 2, 2026-02-20T18:01:22.042 INFO:tasks.workunit.client.0.trial195.stdout: "namespace": "" 2026-02-20T18:01:22.042 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:01:22.042 INFO:tasks.workunit.client.0.trial195.stdout: "need": "47'251", 2026-02-20T18:01:22.042 INFO:tasks.workunit.client.0.trial195.stdout: "have": "0'0", 2026-02-20T18:01:22.042 INFO:tasks.workunit.client.0.trial195.stdout: "flags": "none", 2026-02-20T18:01:22.042 INFO:tasks.workunit.client.0.trial195.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2026-02-20T18:01:22.042 INFO:tasks.workunit.client.0.trial195.stdout: "locations": [ 2026-02-20T18:01:22.042 INFO:tasks.workunit.client.0.trial195.stdout: "0(3)", 2026-02-20T18:01:22.043 INFO:tasks.workunit.client.0.trial195.stdout: "4(2)" 2026-02-20T18:01:22.043 INFO:tasks.workunit.client.0.trial195.stdout: ] 2026-02-20T18:01:22.043 INFO:tasks.workunit.client.0.trial195.stdout: } 2026-02-20T18:01:22.043 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:01:22.043 INFO:tasks.workunit.client.0.trial195.stdout: "state": "NotRecovering", 2026-02-20T18:01:22.043 INFO:tasks.workunit.client.0.trial195.stdout: "available_might_have_unfound": true, 2026-02-20T18:01:22.043 INFO:tasks.workunit.client.0.trial195.stdout: "might_have_unfound": [ 2026-02-20T18:01:22.043 INFO:tasks.workunit.client.0.trial195.stdout: { 2026-02-20T18:01:22.043 INFO:tasks.workunit.client.0.trial195.stdout: "osd": "2(4)", 2026-02-20T18:01:22.043 INFO:tasks.workunit.client.0.trial195.stdout: "status": "osd is down" 2026-02-20T18:01:22.043 INFO:tasks.workunit.client.0.trial195.stdout: } 2026-02-20T18:01:22.043 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:01:22.044 INFO:tasks.workunit.client.0.trial195.stdout: "more": false 2026-02-20T18:01:22.044 INFO:tasks.workunit.client.0.trial195.stdout:} 2026-02-20T18:01:22.045 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:588: TEST_ec_backfill_unfound: ceph pg 2.0 query 2026-02-20T18:01:22.109 INFO:tasks.workunit.client.0.trial195.stdout:{ 2026-02-20T18:01:22.109 INFO:tasks.workunit.client.0.trial195.stdout: "snap_trimq": "[]", 2026-02-20T18:01:22.109 INFO:tasks.workunit.client.0.trial195.stdout: "snap_trimq_len": 0, 2026-02-20T18:01:22.109 INFO:tasks.workunit.client.0.trial195.stdout: "state": "active+recovery_unfound+undersized+degraded", 2026-02-20T18:01:22.109 INFO:tasks.workunit.client.0.trial195.stdout: "epoch": 58, 2026-02-20T18:01:22.109 INFO:tasks.workunit.client.0.trial195.stdout: "up": [ 2026-02-20T18:01:22.109 INFO:tasks.workunit.client.0.trial195.stdout: 3, 2026-02-20T18:01:22.109 INFO:tasks.workunit.client.0.trial195.stdout: 1, 2026-02-20T18:01:22.109 INFO:tasks.workunit.client.0.trial195.stdout: 4, 2026-02-20T18:01:22.110 INFO:tasks.workunit.client.0.trial195.stdout: 0, 2026-02-20T18:01:22.110 INFO:tasks.workunit.client.0.trial195.stdout: 2147483647 2026-02-20T18:01:22.110 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:01:22.110 INFO:tasks.workunit.client.0.trial195.stdout: "acting": [ 2026-02-20T18:01:22.110 INFO:tasks.workunit.client.0.trial195.stdout: 3, 2026-02-20T18:01:22.110 INFO:tasks.workunit.client.0.trial195.stdout: 1, 2026-02-20T18:01:22.110 INFO:tasks.workunit.client.0.trial195.stdout: 4, 2026-02-20T18:01:22.110 INFO:tasks.workunit.client.0.trial195.stdout: 0, 2026-02-20T18:01:22.110 INFO:tasks.workunit.client.0.trial195.stdout: 2147483647 2026-02-20T18:01:22.110 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:01:22.110 INFO:tasks.workunit.client.0.trial195.stdout: "acting_recovery_backfill": [ 2026-02-20T18:01:22.110 INFO:tasks.workunit.client.0.trial195.stdout: "0(3)", 2026-02-20T18:01:22.111 INFO:tasks.workunit.client.0.trial195.stdout: "1(1)", 2026-02-20T18:01:22.111 INFO:tasks.workunit.client.0.trial195.stdout: "3(0)", 2026-02-20T18:01:22.111 INFO:tasks.workunit.client.0.trial195.stdout: "4(2)" 2026-02-20T18:01:22.111 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:01:22.111 INFO:tasks.workunit.client.0.trial195.stdout: "info": { 2026-02-20T18:01:22.111 INFO:tasks.workunit.client.0.trial195.stdout: "pgid": "2.0s0", 2026-02-20T18:01:22.111 INFO:tasks.workunit.client.0.trial195.stdout: "shard": "0", 2026-02-20T18:01:22.111 INFO:tasks.workunit.client.0.trial195.stdout: "last_update": "47'301", 2026-02-20T18:01:22.111 INFO:tasks.workunit.client.0.trial195.stdout: "last_complete": "47'250", 2026-02-20T18:01:22.111 INFO:tasks.workunit.client.0.trial195.stdout: "log_tail": "47'200", 2026-02-20T18:01:22.111 INFO:tasks.workunit.client.0.trial195.stdout: "last_user_version": 301, 2026-02-20T18:01:22.111 INFO:tasks.workunit.client.0.trial195.stdout: "last_backfill": "MAX", 2026-02-20T18:01:22.112 INFO:tasks.workunit.client.0.trial195.stdout: "partial_writes_last_complete": [], 2026-02-20T18:01:22.112 INFO:tasks.workunit.client.0.trial195.stdout: "partial_writes_last_complete_epoch": "57", 2026-02-20T18:01:22.112 INFO:tasks.workunit.client.0.trial195.stdout: "purged_snaps": [], 2026-02-20T18:01:22.112 INFO:tasks.workunit.client.0.trial195.stdout: "history": { 2026-02-20T18:01:22.112 INFO:tasks.workunit.client.0.trial195.stdout: "epoch_created": 41, 2026-02-20T18:01:22.112 INFO:tasks.workunit.client.0.trial195.stdout: "epoch_pool_created": 41, 2026-02-20T18:01:22.112 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_started": 58, 2026-02-20T18:01:22.112 INFO:tasks.workunit.client.0.trial195.stdout: "last_interval_started": 57, 2026-02-20T18:01:22.112 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_clean": 42, 2026-02-20T18:01:22.112 INFO:tasks.workunit.client.0.trial195.stdout: "last_interval_clean": 41, 2026-02-20T18:01:22.112 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_split": 0, 2026-02-20T18:01:22.112 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_marked_full": 0, 2026-02-20T18:01:22.113 INFO:tasks.workunit.client.0.trial195.stdout: "same_up_since": 57, 2026-02-20T18:01:22.113 INFO:tasks.workunit.client.0.trial195.stdout: "same_interval_since": 57, 2026-02-20T18:01:22.113 INFO:tasks.workunit.client.0.trial195.stdout: "same_primary_since": 41, 2026-02-20T18:01:22.113 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub": "0'0", 2026-02-20T18:01:22.113 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub_stamp": "2026-02-20T18:00:46.410154+0000", 2026-02-20T18:01:22.113 INFO:tasks.workunit.client.0.trial195.stdout: "last_deep_scrub": "0'0", 2026-02-20T18:01:22.113 INFO:tasks.workunit.client.0.trial195.stdout: "last_deep_scrub_stamp": "2026-02-20T18:00:46.410154+0000", 2026-02-20T18:01:22.113 INFO:tasks.workunit.client.0.trial195.stdout: "last_clean_scrub_stamp": "2026-02-20T18:00:46.410154+0000", 2026-02-20T18:01:22.113 INFO:tasks.workunit.client.0.trial195.stdout: "prior_readable_until_ub": 0 2026-02-20T18:01:22.113 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:01:22.113 INFO:tasks.workunit.client.0.trial195.stdout: "stats": { 2026-02-20T18:01:22.113 INFO:tasks.workunit.client.0.trial195.stdout: "version": "47'301", 2026-02-20T18:01:22.113 INFO:tasks.workunit.client.0.trial195.stdout: "reported_seq": 922, 2026-02-20T18:01:22.114 INFO:tasks.workunit.client.0.trial195.stdout: "reported_epoch": 58, 2026-02-20T18:01:22.114 INFO:tasks.workunit.client.0.trial195.stdout: "state": "active+recovery_unfound+undersized+degraded", 2026-02-20T18:01:22.114 INFO:tasks.workunit.client.0.trial195.stdout: "last_fresh": "2026-02-20T18:01:16.808051+0000", 2026-02-20T18:01:22.114 INFO:tasks.workunit.client.0.trial195.stdout: "last_change": "2026-02-20T18:01:16.808051+0000", 2026-02-20T18:01:22.114 INFO:tasks.workunit.client.0.trial195.stdout: "last_active": "2026-02-20T18:01:16.808051+0000", 2026-02-20T18:01:22.114 INFO:tasks.workunit.client.0.trial195.stdout: "last_peered": "2026-02-20T18:01:16.808051+0000", 2026-02-20T18:01:22.114 INFO:tasks.workunit.client.0.trial195.stdout: "last_clean": "2026-02-20T18:00:51.565581+0000", 2026-02-20T18:01:22.114 INFO:tasks.workunit.client.0.trial195.stdout: "last_became_active": "2026-02-20T18:01:16.806047+0000", 2026-02-20T18:01:22.114 INFO:tasks.workunit.client.0.trial195.stdout: "last_became_peered": "2026-02-20T18:01:16.806047+0000", 2026-02-20T18:01:22.114 INFO:tasks.workunit.client.0.trial195.stdout: "last_unstale": "2026-02-20T18:01:16.808051+0000", 2026-02-20T18:01:22.114 INFO:tasks.workunit.client.0.trial195.stdout: "last_undegraded": "2026-02-20T18:01:16.804570+0000", 2026-02-20T18:01:22.114 INFO:tasks.workunit.client.0.trial195.stdout: "last_fullsized": "2026-02-20T18:01:16.804376+0000", 2026-02-20T18:01:22.115 INFO:tasks.workunit.client.0.trial195.stdout: "mapping_epoch": 57, 2026-02-20T18:01:22.115 INFO:tasks.workunit.client.0.trial195.stdout: "log_start": "47'200", 2026-02-20T18:01:22.115 INFO:tasks.workunit.client.0.trial195.stdout: "ondisk_log_start": "47'200", 2026-02-20T18:01:22.115 INFO:tasks.workunit.client.0.trial195.stdout: "created": 41, 2026-02-20T18:01:22.115 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_clean": 42, 2026-02-20T18:01:22.115 INFO:tasks.workunit.client.0.trial195.stdout: "parent": "0.0", 2026-02-20T18:01:22.115 INFO:tasks.workunit.client.0.trial195.stdout: "parent_split_bits": 0, 2026-02-20T18:01:22.115 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub": "0'0", 2026-02-20T18:01:22.115 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub_stamp": "2026-02-20T18:00:46.410154+0000", 2026-02-20T18:01:22.115 INFO:tasks.workunit.client.0.trial195.stdout: "last_deep_scrub": "0'0", 2026-02-20T18:01:22.115 INFO:tasks.workunit.client.0.trial195.stdout: "last_deep_scrub_stamp": "2026-02-20T18:00:46.410154+0000", 2026-02-20T18:01:22.115 INFO:tasks.workunit.client.0.trial195.stdout: "last_clean_scrub_stamp": "2026-02-20T18:00:46.410154+0000", 2026-02-20T18:01:22.116 INFO:tasks.workunit.client.0.trial195.stdout: "objects_scrubbed": 0, 2026-02-20T18:01:22.116 INFO:tasks.workunit.client.0.trial195.stdout: "log_size": 101, 2026-02-20T18:01:22.116 INFO:tasks.workunit.client.0.trial195.stdout: "log_dups_size": 200, 2026-02-20T18:01:22.116 INFO:tasks.workunit.client.0.trial195.stdout: "ondisk_log_size": 101, 2026-02-20T18:01:22.116 INFO:tasks.workunit.client.0.trial195.stdout: "stats_invalid": false, 2026-02-20T18:01:22.116 INFO:tasks.workunit.client.0.trial195.stdout: "dirty_stats_invalid": false, 2026-02-20T18:01:22.116 INFO:tasks.workunit.client.0.trial195.stdout: "omap_stats_invalid": false, 2026-02-20T18:01:22.116 INFO:tasks.workunit.client.0.trial195.stdout: "hitset_stats_invalid": false, 2026-02-20T18:01:22.116 INFO:tasks.workunit.client.0.trial195.stdout: "hitset_bytes_stats_invalid": false, 2026-02-20T18:01:22.116 INFO:tasks.workunit.client.0.trial195.stdout: "pin_stats_invalid": false, 2026-02-20T18:01:22.116 INFO:tasks.workunit.client.0.trial195.stdout: "manifest_stats_invalid": false, 2026-02-20T18:01:22.116 INFO:tasks.workunit.client.0.trial195.stdout: "snaptrimq_len": 0, 2026-02-20T18:01:22.116 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub_duration": 0, 2026-02-20T18:01:22.117 INFO:tasks.workunit.client.0.trial195.stdout: "scrub_schedule": "no scrub is scheduled", 2026-02-20T18:01:22.117 INFO:tasks.workunit.client.0.trial195.stdout: "scrub_duration": 0, 2026-02-20T18:01:22.117 INFO:tasks.workunit.client.0.trial195.stdout: "objects_trimmed": 0, 2026-02-20T18:01:22.117 INFO:tasks.workunit.client.0.trial195.stdout: "snaptrim_duration": 0, 2026-02-20T18:01:22.117 INFO:tasks.workunit.client.0.trial195.stdout: "stat_sum": { 2026-02-20T18:01:22.117 INFO:tasks.workunit.client.0.trial195.stdout: "num_bytes": 1232896, 2026-02-20T18:01:22.117 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects": 301, 2026-02-20T18:01:22.117 INFO:tasks.workunit.client.0.trial195.stdout: "num_object_clones": 0, 2026-02-20T18:01:22.117 INFO:tasks.workunit.client.0.trial195.stdout: "num_object_copies": 1505, 2026-02-20T18:01:22.117 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_missing_on_primary": 1, 2026-02-20T18:01:22.117 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_missing": 1, 2026-02-20T18:01:22.117 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_degraded": 303, 2026-02-20T18:01:22.117 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_misplaced": 0, 2026-02-20T18:01:22.118 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_unfound": 1, 2026-02-20T18:01:22.118 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_dirty": 301, 2026-02-20T18:01:22.118 INFO:tasks.workunit.client.0.trial195.stdout: "num_whiteouts": 0, 2026-02-20T18:01:22.118 INFO:tasks.workunit.client.0.trial195.stdout: "num_read": 0, 2026-02-20T18:01:22.118 INFO:tasks.workunit.client.0.trial195.stdout: "num_read_kb": 0, 2026-02-20T18:01:22.118 INFO:tasks.workunit.client.0.trial195.stdout: "num_write": 301, 2026-02-20T18:01:22.118 INFO:tasks.workunit.client.0.trial195.stdout: "num_write_kb": 1204, 2026-02-20T18:01:22.118 INFO:tasks.workunit.client.0.trial195.stdout: "num_scrub_errors": 0, 2026-02-20T18:01:22.118 INFO:tasks.workunit.client.0.trial195.stdout: "num_shallow_scrub_errors": 0, 2026-02-20T18:01:22.118 INFO:tasks.workunit.client.0.trial195.stdout: "num_deep_scrub_errors": 0, 2026-02-20T18:01:22.118 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_recovered": 0, 2026-02-20T18:01:22.118 INFO:tasks.workunit.client.0.trial195.stdout: "num_bytes_recovered": 0, 2026-02-20T18:01:22.118 INFO:tasks.workunit.client.0.trial195.stdout: "num_keys_recovered": 0, 2026-02-20T18:01:22.119 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_omap": 0, 2026-02-20T18:01:22.119 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_hit_set_archive": 0, 2026-02-20T18:01:22.119 INFO:tasks.workunit.client.0.trial195.stdout: "num_bytes_hit_set_archive": 0, 2026-02-20T18:01:22.119 INFO:tasks.workunit.client.0.trial195.stdout: "num_flush": 0, 2026-02-20T18:01:22.119 INFO:tasks.workunit.client.0.trial195.stdout: "num_flush_kb": 0, 2026-02-20T18:01:22.119 INFO:tasks.workunit.client.0.trial195.stdout: "num_evict": 0, 2026-02-20T18:01:22.119 INFO:tasks.workunit.client.0.trial195.stdout: "num_evict_kb": 0, 2026-02-20T18:01:22.119 INFO:tasks.workunit.client.0.trial195.stdout: "num_promote": 0, 2026-02-20T18:01:22.119 INFO:tasks.workunit.client.0.trial195.stdout: "num_flush_mode_high": 0, 2026-02-20T18:01:22.119 INFO:tasks.workunit.client.0.trial195.stdout: "num_flush_mode_low": 0, 2026-02-20T18:01:22.119 INFO:tasks.workunit.client.0.trial195.stdout: "num_evict_mode_some": 0, 2026-02-20T18:01:22.119 INFO:tasks.workunit.client.0.trial195.stdout: "num_evict_mode_full": 0, 2026-02-20T18:01:22.119 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_pinned": 0, 2026-02-20T18:01:22.120 INFO:tasks.workunit.client.0.trial195.stdout: "num_legacy_snapsets": 0, 2026-02-20T18:01:22.120 INFO:tasks.workunit.client.0.trial195.stdout: "num_large_omap_objects": 0, 2026-02-20T18:01:22.120 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_manifest": 0, 2026-02-20T18:01:22.120 INFO:tasks.workunit.client.0.trial195.stdout: "num_omap_bytes": 0, 2026-02-20T18:01:22.120 INFO:tasks.workunit.client.0.trial195.stdout: "num_omap_keys": 0, 2026-02-20T18:01:22.120 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_repaired": 0 2026-02-20T18:01:22.120 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:01:22.120 INFO:tasks.workunit.client.0.trial195.stdout: "up": [ 2026-02-20T18:01:22.120 INFO:tasks.workunit.client.0.trial195.stdout: 3, 2026-02-20T18:01:22.120 INFO:tasks.workunit.client.0.trial195.stdout: 1, 2026-02-20T18:01:22.120 INFO:tasks.workunit.client.0.trial195.stdout: 4, 2026-02-20T18:01:22.120 INFO:tasks.workunit.client.0.trial195.stdout: 0, 2026-02-20T18:01:22.121 INFO:tasks.workunit.client.0.trial195.stdout: 2147483647 2026-02-20T18:01:22.121 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:01:22.121 INFO:tasks.workunit.client.0.trial195.stdout: "acting": [ 2026-02-20T18:01:22.121 INFO:tasks.workunit.client.0.trial195.stdout: 3, 2026-02-20T18:01:22.121 INFO:tasks.workunit.client.0.trial195.stdout: 1, 2026-02-20T18:01:22.121 INFO:tasks.workunit.client.0.trial195.stdout: 4, 2026-02-20T18:01:22.121 INFO:tasks.workunit.client.0.trial195.stdout: 0, 2026-02-20T18:01:22.121 INFO:tasks.workunit.client.0.trial195.stdout: 2147483647 2026-02-20T18:01:22.121 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:01:22.121 INFO:tasks.workunit.client.0.trial195.stdout: "avail_no_missing": [ 2026-02-20T18:01:22.121 INFO:tasks.workunit.client.0.trial195.stdout: "0(3)", 2026-02-20T18:01:22.121 INFO:tasks.workunit.client.0.trial195.stdout: "4(2)" 2026-02-20T18:01:22.121 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:01:22.122 INFO:tasks.workunit.client.0.trial195.stdout: "object_location_counts": [ 2026-02-20T18:01:22.122 INFO:tasks.workunit.client.0.trial195.stdout: { 2026-02-20T18:01:22.122 INFO:tasks.workunit.client.0.trial195.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-20T18:01:22.122 INFO:tasks.workunit.client.0.trial195.stdout: "objects": 300 2026-02-20T18:01:22.122 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:01:22.122 INFO:tasks.workunit.client.0.trial195.stdout: { 2026-02-20T18:01:22.122 INFO:tasks.workunit.client.0.trial195.stdout: "shards": "0(3),4(2)", 2026-02-20T18:01:22.122 INFO:tasks.workunit.client.0.trial195.stdout: "objects": 1 2026-02-20T18:01:22.122 INFO:tasks.workunit.client.0.trial195.stdout: } 2026-02-20T18:01:22.122 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:01:22.122 INFO:tasks.workunit.client.0.trial195.stdout: "blocked_by": [], 2026-02-20T18:01:22.122 INFO:tasks.workunit.client.0.trial195.stdout: "up_primary": 3, 2026-02-20T18:01:22.123 INFO:tasks.workunit.client.0.trial195.stdout: "acting_primary": 3, 2026-02-20T18:01:22.123 INFO:tasks.workunit.client.0.trial195.stdout: "purged_snaps": [] 2026-02-20T18:01:22.123 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:01:22.123 INFO:tasks.workunit.client.0.trial195.stdout: "empty": 0, 2026-02-20T18:01:22.123 INFO:tasks.workunit.client.0.trial195.stdout: "dne": 0, 2026-02-20T18:01:22.123 INFO:tasks.workunit.client.0.trial195.stdout: "incomplete": 0, 2026-02-20T18:01:22.123 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_started": 58, 2026-02-20T18:01:22.123 INFO:tasks.workunit.client.0.trial195.stdout: "hit_set_history": { 2026-02-20T18:01:22.123 INFO:tasks.workunit.client.0.trial195.stdout: "current_last_update": "0'0", 2026-02-20T18:01:22.123 INFO:tasks.workunit.client.0.trial195.stdout: "history": [] 2026-02-20T18:01:22.123 INFO:tasks.workunit.client.0.trial195.stdout: } 2026-02-20T18:01:22.123 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:01:22.123 INFO:tasks.workunit.client.0.trial195.stdout: "peer_info": [ 2026-02-20T18:01:22.124 INFO:tasks.workunit.client.0.trial195.stdout: { 2026-02-20T18:01:22.124 INFO:tasks.workunit.client.0.trial195.stdout: "peer": "0(3)", 2026-02-20T18:01:22.124 INFO:tasks.workunit.client.0.trial195.stdout: "pgid": "2.0s3", 2026-02-20T18:01:22.124 INFO:tasks.workunit.client.0.trial195.stdout: "shard": "3", 2026-02-20T18:01:22.124 INFO:tasks.workunit.client.0.trial195.stdout: "last_update": "47'301", 2026-02-20T18:01:22.124 INFO:tasks.workunit.client.0.trial195.stdout: "last_complete": "47'301", 2026-02-20T18:01:22.124 INFO:tasks.workunit.client.0.trial195.stdout: "log_tail": "47'200", 2026-02-20T18:01:22.124 INFO:tasks.workunit.client.0.trial195.stdout: "last_user_version": 301, 2026-02-20T18:01:22.124 INFO:tasks.workunit.client.0.trial195.stdout: "last_backfill": "MAX", 2026-02-20T18:01:22.124 INFO:tasks.workunit.client.0.trial195.stdout: "partial_writes_last_complete": [], 2026-02-20T18:01:22.124 INFO:tasks.workunit.client.0.trial195.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-20T18:01:22.124 INFO:tasks.workunit.client.0.trial195.stdout: "purged_snaps": [], 2026-02-20T18:01:22.124 INFO:tasks.workunit.client.0.trial195.stdout: "history": { 2026-02-20T18:01:22.125 INFO:tasks.workunit.client.0.trial195.stdout: "epoch_created": 41, 2026-02-20T18:01:22.125 INFO:tasks.workunit.client.0.trial195.stdout: "epoch_pool_created": 41, 2026-02-20T18:01:22.125 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_started": 58, 2026-02-20T18:01:22.125 INFO:tasks.workunit.client.0.trial195.stdout: "last_interval_started": 57, 2026-02-20T18:01:22.125 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_clean": 42, 2026-02-20T18:01:22.125 INFO:tasks.workunit.client.0.trial195.stdout: "last_interval_clean": 41, 2026-02-20T18:01:22.125 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_split": 0, 2026-02-20T18:01:22.125 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_marked_full": 0, 2026-02-20T18:01:22.125 INFO:tasks.workunit.client.0.trial195.stdout: "same_up_since": 57, 2026-02-20T18:01:22.125 INFO:tasks.workunit.client.0.trial195.stdout: "same_interval_since": 57, 2026-02-20T18:01:22.125 INFO:tasks.workunit.client.0.trial195.stdout: "same_primary_since": 41, 2026-02-20T18:01:22.125 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub": "0'0", 2026-02-20T18:01:22.125 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub_stamp": "2026-02-20T18:00:46.410154+0000", 2026-02-20T18:01:22.126 INFO:tasks.workunit.client.0.trial195.stdout: "last_deep_scrub": "0'0", 2026-02-20T18:01:22.126 INFO:tasks.workunit.client.0.trial195.stdout: "last_deep_scrub_stamp": "2026-02-20T18:00:46.410154+0000", 2026-02-20T18:01:22.126 INFO:tasks.workunit.client.0.trial195.stdout: "last_clean_scrub_stamp": "2026-02-20T18:00:46.410154+0000", 2026-02-20T18:01:22.126 INFO:tasks.workunit.client.0.trial195.stdout: "prior_readable_until_ub": 0 2026-02-20T18:01:22.126 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:01:22.126 INFO:tasks.workunit.client.0.trial195.stdout: "stats": { 2026-02-20T18:01:22.126 INFO:tasks.workunit.client.0.trial195.stdout: "version": "47'301", 2026-02-20T18:01:22.126 INFO:tasks.workunit.client.0.trial195.stdout: "reported_seq": 912, 2026-02-20T18:01:22.126 INFO:tasks.workunit.client.0.trial195.stdout: "reported_epoch": 57, 2026-02-20T18:01:22.126 INFO:tasks.workunit.client.0.trial195.stdout: "state": "active+undersized+degraded", 2026-02-20T18:01:22.126 INFO:tasks.workunit.client.0.trial195.stdout: "last_fresh": "2026-02-20T18:00:58.575617+0000", 2026-02-20T18:01:22.126 INFO:tasks.workunit.client.0.trial195.stdout: "last_change": "2026-02-20T18:00:52.178278+0000", 2026-02-20T18:01:22.127 INFO:tasks.workunit.client.0.trial195.stdout: "last_active": "2026-02-20T18:00:58.575617+0000", 2026-02-20T18:01:22.127 INFO:tasks.workunit.client.0.trial195.stdout: "last_peered": "2026-02-20T18:00:58.575617+0000", 2026-02-20T18:01:22.127 INFO:tasks.workunit.client.0.trial195.stdout: "last_clean": "2026-02-20T18:00:51.565581+0000", 2026-02-20T18:01:22.127 INFO:tasks.workunit.client.0.trial195.stdout: "last_became_active": "2026-02-20T18:00:52.178278+0000", 2026-02-20T18:01:22.127 INFO:tasks.workunit.client.0.trial195.stdout: "last_became_peered": "2026-02-20T18:00:52.178278+0000", 2026-02-20T18:01:22.127 INFO:tasks.workunit.client.0.trial195.stdout: "last_unstale": "2026-02-20T18:00:58.575617+0000", 2026-02-20T18:01:22.127 INFO:tasks.workunit.client.0.trial195.stdout: "last_undegraded": "2026-02-20T18:00:52.176952+0000", 2026-02-20T18:01:22.127 INFO:tasks.workunit.client.0.trial195.stdout: "last_fullsized": "2026-02-20T18:00:52.176862+0000", 2026-02-20T18:01:22.127 INFO:tasks.workunit.client.0.trial195.stdout: "mapping_epoch": 57, 2026-02-20T18:01:22.127 INFO:tasks.workunit.client.0.trial195.stdout: "log_start": "47'200", 2026-02-20T18:01:22.127 INFO:tasks.workunit.client.0.trial195.stdout: "ondisk_log_start": "47'200", 2026-02-20T18:01:22.127 INFO:tasks.workunit.client.0.trial195.stdout: "created": 41, 2026-02-20T18:01:22.128 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_clean": 42, 2026-02-20T18:01:22.128 INFO:tasks.workunit.client.0.trial195.stdout: "parent": "0.0", 2026-02-20T18:01:22.128 INFO:tasks.workunit.client.0.trial195.stdout: "parent_split_bits": 0, 2026-02-20T18:01:22.128 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub": "0'0", 2026-02-20T18:01:22.128 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub_stamp": "2026-02-20T18:00:46.410154+0000", 2026-02-20T18:01:22.128 INFO:tasks.workunit.client.0.trial195.stdout: "last_deep_scrub": "0'0", 2026-02-20T18:01:22.128 INFO:tasks.workunit.client.0.trial195.stdout: "last_deep_scrub_stamp": "2026-02-20T18:00:46.410154+0000", 2026-02-20T18:01:22.128 INFO:tasks.workunit.client.0.trial195.stdout: "last_clean_scrub_stamp": "2026-02-20T18:00:46.410154+0000", 2026-02-20T18:01:22.128 INFO:tasks.workunit.client.0.trial195.stdout: "objects_scrubbed": 0, 2026-02-20T18:01:22.128 INFO:tasks.workunit.client.0.trial195.stdout: "log_size": 101, 2026-02-20T18:01:22.128 INFO:tasks.workunit.client.0.trial195.stdout: "log_dups_size": 200, 2026-02-20T18:01:22.128 INFO:tasks.workunit.client.0.trial195.stdout: "ondisk_log_size": 101, 2026-02-20T18:01:22.128 INFO:tasks.workunit.client.0.trial195.stdout: "stats_invalid": false, 2026-02-20T18:01:22.129 INFO:tasks.workunit.client.0.trial195.stdout: "dirty_stats_invalid": false, 2026-02-20T18:01:22.129 INFO:tasks.workunit.client.0.trial195.stdout: "omap_stats_invalid": false, 2026-02-20T18:01:22.129 INFO:tasks.workunit.client.0.trial195.stdout: "hitset_stats_invalid": false, 2026-02-20T18:01:22.129 INFO:tasks.workunit.client.0.trial195.stdout: "hitset_bytes_stats_invalid": false, 2026-02-20T18:01:22.129 INFO:tasks.workunit.client.0.trial195.stdout: "pin_stats_invalid": false, 2026-02-20T18:01:22.129 INFO:tasks.workunit.client.0.trial195.stdout: "manifest_stats_invalid": false, 2026-02-20T18:01:22.129 INFO:tasks.workunit.client.0.trial195.stdout: "snaptrimq_len": 0, 2026-02-20T18:01:22.129 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub_duration": 0, 2026-02-20T18:01:22.129 INFO:tasks.workunit.client.0.trial195.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-22T02:50:06.103041+0000", 2026-02-20T18:01:22.129 INFO:tasks.workunit.client.0.trial195.stdout: "scrub_duration": 0, 2026-02-20T18:01:22.129 INFO:tasks.workunit.client.0.trial195.stdout: "objects_trimmed": 0, 2026-02-20T18:01:22.129 INFO:tasks.workunit.client.0.trial195.stdout: "snaptrim_duration": 0, 2026-02-20T18:01:22.129 INFO:tasks.workunit.client.0.trial195.stdout: "stat_sum": { 2026-02-20T18:01:22.130 INFO:tasks.workunit.client.0.trial195.stdout: "num_bytes": 1232896, 2026-02-20T18:01:22.130 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects": 301, 2026-02-20T18:01:22.130 INFO:tasks.workunit.client.0.trial195.stdout: "num_object_clones": 0, 2026-02-20T18:01:22.130 INFO:tasks.workunit.client.0.trial195.stdout: "num_object_copies": 1505, 2026-02-20T18:01:22.130 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_missing_on_primary": 0, 2026-02-20T18:01:22.130 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_missing": 0, 2026-02-20T18:01:22.130 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_degraded": 301, 2026-02-20T18:01:22.130 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_misplaced": 0, 2026-02-20T18:01:22.130 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_unfound": 0, 2026-02-20T18:01:22.130 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_dirty": 301, 2026-02-20T18:01:22.130 INFO:tasks.workunit.client.0.trial195.stdout: "num_whiteouts": 0, 2026-02-20T18:01:22.130 INFO:tasks.workunit.client.0.trial195.stdout: "num_read": 0, 2026-02-20T18:01:22.131 INFO:tasks.workunit.client.0.trial195.stdout: "num_read_kb": 0, 2026-02-20T18:01:22.131 INFO:tasks.workunit.client.0.trial195.stdout: "num_write": 301, 2026-02-20T18:01:22.131 INFO:tasks.workunit.client.0.trial195.stdout: "num_write_kb": 1204, 2026-02-20T18:01:22.131 INFO:tasks.workunit.client.0.trial195.stdout: "num_scrub_errors": 0, 2026-02-20T18:01:22.131 INFO:tasks.workunit.client.0.trial195.stdout: "num_shallow_scrub_errors": 0, 2026-02-20T18:01:22.131 INFO:tasks.workunit.client.0.trial195.stdout: "num_deep_scrub_errors": 0, 2026-02-20T18:01:22.131 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_recovered": 0, 2026-02-20T18:01:22.131 INFO:tasks.workunit.client.0.trial195.stdout: "num_bytes_recovered": 0, 2026-02-20T18:01:22.131 INFO:tasks.workunit.client.0.trial195.stdout: "num_keys_recovered": 0, 2026-02-20T18:01:22.131 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_omap": 0, 2026-02-20T18:01:22.131 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_hit_set_archive": 0, 2026-02-20T18:01:22.131 INFO:tasks.workunit.client.0.trial195.stdout: "num_bytes_hit_set_archive": 0, 2026-02-20T18:01:22.131 INFO:tasks.workunit.client.0.trial195.stdout: "num_flush": 0, 2026-02-20T18:01:22.132 INFO:tasks.workunit.client.0.trial195.stdout: "num_flush_kb": 0, 2026-02-20T18:01:22.132 INFO:tasks.workunit.client.0.trial195.stdout: "num_evict": 0, 2026-02-20T18:01:22.132 INFO:tasks.workunit.client.0.trial195.stdout: "num_evict_kb": 0, 2026-02-20T18:01:22.132 INFO:tasks.workunit.client.0.trial195.stdout: "num_promote": 0, 2026-02-20T18:01:22.132 INFO:tasks.workunit.client.0.trial195.stdout: "num_flush_mode_high": 0, 2026-02-20T18:01:22.132 INFO:tasks.workunit.client.0.trial195.stdout: "num_flush_mode_low": 0, 2026-02-20T18:01:22.132 INFO:tasks.workunit.client.0.trial195.stdout: "num_evict_mode_some": 0, 2026-02-20T18:01:22.132 INFO:tasks.workunit.client.0.trial195.stdout: "num_evict_mode_full": 0, 2026-02-20T18:01:22.132 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_pinned": 0, 2026-02-20T18:01:22.132 INFO:tasks.workunit.client.0.trial195.stdout: "num_legacy_snapsets": 0, 2026-02-20T18:01:22.132 INFO:tasks.workunit.client.0.trial195.stdout: "num_large_omap_objects": 0, 2026-02-20T18:01:22.132 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_manifest": 0, 2026-02-20T18:01:22.132 INFO:tasks.workunit.client.0.trial195.stdout: "num_omap_bytes": 0, 2026-02-20T18:01:22.133 INFO:tasks.workunit.client.0.trial195.stdout: "num_omap_keys": 0, 2026-02-20T18:01:22.133 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_repaired": 0 2026-02-20T18:01:22.133 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:01:22.133 INFO:tasks.workunit.client.0.trial195.stdout: "up": [ 2026-02-20T18:01:22.133 INFO:tasks.workunit.client.0.trial195.stdout: 3, 2026-02-20T18:01:22.133 INFO:tasks.workunit.client.0.trial195.stdout: 1, 2026-02-20T18:01:22.133 INFO:tasks.workunit.client.0.trial195.stdout: 4, 2026-02-20T18:01:22.133 INFO:tasks.workunit.client.0.trial195.stdout: 0, 2026-02-20T18:01:22.133 INFO:tasks.workunit.client.0.trial195.stdout: 2147483647 2026-02-20T18:01:22.133 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:01:22.133 INFO:tasks.workunit.client.0.trial195.stdout: "acting": [ 2026-02-20T18:01:22.133 INFO:tasks.workunit.client.0.trial195.stdout: 3, 2026-02-20T18:01:22.133 INFO:tasks.workunit.client.0.trial195.stdout: 1, 2026-02-20T18:01:22.134 INFO:tasks.workunit.client.0.trial195.stdout: 4, 2026-02-20T18:01:22.134 INFO:tasks.workunit.client.0.trial195.stdout: 0, 2026-02-20T18:01:22.134 INFO:tasks.workunit.client.0.trial195.stdout: 2147483647 2026-02-20T18:01:22.134 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:01:22.134 INFO:tasks.workunit.client.0.trial195.stdout: "avail_no_missing": [ 2026-02-20T18:01:22.134 INFO:tasks.workunit.client.0.trial195.stdout: "3(0)", 2026-02-20T18:01:22.134 INFO:tasks.workunit.client.0.trial195.stdout: "0(3)", 2026-02-20T18:01:22.134 INFO:tasks.workunit.client.0.trial195.stdout: "1(1)", 2026-02-20T18:01:22.134 INFO:tasks.workunit.client.0.trial195.stdout: "4(2)" 2026-02-20T18:01:22.134 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:01:22.135 INFO:tasks.workunit.client.0.trial195.stdout: "object_location_counts": [ 2026-02-20T18:01:22.135 INFO:tasks.workunit.client.0.trial195.stdout: { 2026-02-20T18:01:22.135 INFO:tasks.workunit.client.0.trial195.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-20T18:01:22.135 INFO:tasks.workunit.client.0.trial195.stdout: "objects": 301 2026-02-20T18:01:22.135 INFO:tasks.workunit.client.0.trial195.stdout: } 2026-02-20T18:01:22.135 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:01:22.135 INFO:tasks.workunit.client.0.trial195.stdout: "blocked_by": [], 2026-02-20T18:01:22.135 INFO:tasks.workunit.client.0.trial195.stdout: "up_primary": 3, 2026-02-20T18:01:22.135 INFO:tasks.workunit.client.0.trial195.stdout: "acting_primary": 3, 2026-02-20T18:01:22.135 INFO:tasks.workunit.client.0.trial195.stdout: "purged_snaps": [] 2026-02-20T18:01:22.136 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:01:22.136 INFO:tasks.workunit.client.0.trial195.stdout: "empty": 0, 2026-02-20T18:01:22.136 INFO:tasks.workunit.client.0.trial195.stdout: "dne": 0, 2026-02-20T18:01:22.136 INFO:tasks.workunit.client.0.trial195.stdout: "incomplete": 0, 2026-02-20T18:01:22.136 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_started": 58, 2026-02-20T18:01:22.136 INFO:tasks.workunit.client.0.trial195.stdout: "hit_set_history": { 2026-02-20T18:01:22.136 INFO:tasks.workunit.client.0.trial195.stdout: "current_last_update": "0'0", 2026-02-20T18:01:22.136 INFO:tasks.workunit.client.0.trial195.stdout: "history": [] 2026-02-20T18:01:22.136 INFO:tasks.workunit.client.0.trial195.stdout: } 2026-02-20T18:01:22.136 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:01:22.136 INFO:tasks.workunit.client.0.trial195.stdout: { 2026-02-20T18:01:22.136 INFO:tasks.workunit.client.0.trial195.stdout: "peer": "1(1)", 2026-02-20T18:01:22.136 INFO:tasks.workunit.client.0.trial195.stdout: "pgid": "2.0s1", 2026-02-20T18:01:22.137 INFO:tasks.workunit.client.0.trial195.stdout: "shard": "1", 2026-02-20T18:01:22.137 INFO:tasks.workunit.client.0.trial195.stdout: "last_update": "47'301", 2026-02-20T18:01:22.137 INFO:tasks.workunit.client.0.trial195.stdout: "last_complete": "47'301", 2026-02-20T18:01:22.137 INFO:tasks.workunit.client.0.trial195.stdout: "log_tail": "47'200", 2026-02-20T18:01:22.137 INFO:tasks.workunit.client.0.trial195.stdout: "last_user_version": 301, 2026-02-20T18:01:22.137 INFO:tasks.workunit.client.0.trial195.stdout: "last_backfill": "MAX", 2026-02-20T18:01:22.137 INFO:tasks.workunit.client.0.trial195.stdout: "partial_writes_last_complete": [], 2026-02-20T18:01:22.137 INFO:tasks.workunit.client.0.trial195.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-20T18:01:22.137 INFO:tasks.workunit.client.0.trial195.stdout: "purged_snaps": [], 2026-02-20T18:01:22.137 INFO:tasks.workunit.client.0.trial195.stdout: "history": { 2026-02-20T18:01:22.137 INFO:tasks.workunit.client.0.trial195.stdout: "epoch_created": 41, 2026-02-20T18:01:22.137 INFO:tasks.workunit.client.0.trial195.stdout: "epoch_pool_created": 41, 2026-02-20T18:01:22.138 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_started": 58, 2026-02-20T18:01:22.138 INFO:tasks.workunit.client.0.trial195.stdout: "last_interval_started": 57, 2026-02-20T18:01:22.138 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_clean": 42, 2026-02-20T18:01:22.138 INFO:tasks.workunit.client.0.trial195.stdout: "last_interval_clean": 41, 2026-02-20T18:01:22.138 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_split": 0, 2026-02-20T18:01:22.138 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_marked_full": 0, 2026-02-20T18:01:22.138 INFO:tasks.workunit.client.0.trial195.stdout: "same_up_since": 57, 2026-02-20T18:01:22.138 INFO:tasks.workunit.client.0.trial195.stdout: "same_interval_since": 57, 2026-02-20T18:01:22.138 INFO:tasks.workunit.client.0.trial195.stdout: "same_primary_since": 41, 2026-02-20T18:01:22.138 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub": "0'0", 2026-02-20T18:01:22.138 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub_stamp": "2026-02-20T18:00:46.410154+0000", 2026-02-20T18:01:22.138 INFO:tasks.workunit.client.0.trial195.stdout: "last_deep_scrub": "0'0", 2026-02-20T18:01:22.138 INFO:tasks.workunit.client.0.trial195.stdout: "last_deep_scrub_stamp": "2026-02-20T18:00:46.410154+0000", 2026-02-20T18:01:22.138 INFO:tasks.workunit.client.0.trial195.stdout: "last_clean_scrub_stamp": "2026-02-20T18:00:46.410154+0000", 2026-02-20T18:01:22.139 INFO:tasks.workunit.client.0.trial195.stdout: "prior_readable_until_ub": 0 2026-02-20T18:01:22.139 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:01:22.139 INFO:tasks.workunit.client.0.trial195.stdout: "stats": { 2026-02-20T18:01:22.139 INFO:tasks.workunit.client.0.trial195.stdout: "version": "47'301", 2026-02-20T18:01:22.139 INFO:tasks.workunit.client.0.trial195.stdout: "reported_seq": 332, 2026-02-20T18:01:22.139 INFO:tasks.workunit.client.0.trial195.stdout: "reported_epoch": 47, 2026-02-20T18:01:22.139 INFO:tasks.workunit.client.0.trial195.stdout: "state": "active+undersized+degraded", 2026-02-20T18:01:22.139 INFO:tasks.workunit.client.0.trial195.stdout: "last_fresh": "2026-02-20T18:00:58.575617+0000", 2026-02-20T18:01:22.139 INFO:tasks.workunit.client.0.trial195.stdout: "last_change": "2026-02-20T18:00:52.178278+0000", 2026-02-20T18:01:22.139 INFO:tasks.workunit.client.0.trial195.stdout: "last_active": "2026-02-20T18:00:58.575617+0000", 2026-02-20T18:01:22.139 INFO:tasks.workunit.client.0.trial195.stdout: "last_peered": "2026-02-20T18:00:58.575617+0000", 2026-02-20T18:01:22.139 INFO:tasks.workunit.client.0.trial195.stdout: "last_clean": "2026-02-20T18:00:51.565581+0000", 2026-02-20T18:01:22.140 INFO:tasks.workunit.client.0.trial195.stdout: "last_became_active": "2026-02-20T18:00:52.178278+0000", 2026-02-20T18:01:22.140 INFO:tasks.workunit.client.0.trial195.stdout: "last_became_peered": "2026-02-20T18:00:52.178278+0000", 2026-02-20T18:01:22.140 INFO:tasks.workunit.client.0.trial195.stdout: "last_unstale": "2026-02-20T18:00:58.575617+0000", 2026-02-20T18:01:22.140 INFO:tasks.workunit.client.0.trial195.stdout: "last_undegraded": "2026-02-20T18:00:52.176952+0000", 2026-02-20T18:01:22.140 INFO:tasks.workunit.client.0.trial195.stdout: "last_fullsized": "2026-02-20T18:00:52.176862+0000", 2026-02-20T18:01:22.140 INFO:tasks.workunit.client.0.trial195.stdout: "mapping_epoch": 57, 2026-02-20T18:01:22.140 INFO:tasks.workunit.client.0.trial195.stdout: "log_start": "47'200", 2026-02-20T18:01:22.140 INFO:tasks.workunit.client.0.trial195.stdout: "ondisk_log_start": "47'200", 2026-02-20T18:01:22.140 INFO:tasks.workunit.client.0.trial195.stdout: "created": 41, 2026-02-20T18:01:22.140 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_clean": 42, 2026-02-20T18:01:22.140 INFO:tasks.workunit.client.0.trial195.stdout: "parent": "0.0", 2026-02-20T18:01:22.140 INFO:tasks.workunit.client.0.trial195.stdout: "parent_split_bits": 0, 2026-02-20T18:01:22.140 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub": "0'0", 2026-02-20T18:01:22.141 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub_stamp": "2026-02-20T18:00:46.410154+0000", 2026-02-20T18:01:22.141 INFO:tasks.workunit.client.0.trial195.stdout: "last_deep_scrub": "0'0", 2026-02-20T18:01:22.141 INFO:tasks.workunit.client.0.trial195.stdout: "last_deep_scrub_stamp": "2026-02-20T18:00:46.410154+0000", 2026-02-20T18:01:22.141 INFO:tasks.workunit.client.0.trial195.stdout: "last_clean_scrub_stamp": "2026-02-20T18:00:46.410154+0000", 2026-02-20T18:01:22.141 INFO:tasks.workunit.client.0.trial195.stdout: "objects_scrubbed": 0, 2026-02-20T18:01:22.141 INFO:tasks.workunit.client.0.trial195.stdout: "log_size": 101, 2026-02-20T18:01:22.141 INFO:tasks.workunit.client.0.trial195.stdout: "log_dups_size": 200, 2026-02-20T18:01:22.141 INFO:tasks.workunit.client.0.trial195.stdout: "ondisk_log_size": 101, 2026-02-20T18:01:22.141 INFO:tasks.workunit.client.0.trial195.stdout: "stats_invalid": false, 2026-02-20T18:01:22.141 INFO:tasks.workunit.client.0.trial195.stdout: "dirty_stats_invalid": false, 2026-02-20T18:01:22.141 INFO:tasks.workunit.client.0.trial195.stdout: "omap_stats_invalid": false, 2026-02-20T18:01:22.141 INFO:tasks.workunit.client.0.trial195.stdout: "hitset_stats_invalid": false, 2026-02-20T18:01:22.141 INFO:tasks.workunit.client.0.trial195.stdout: "hitset_bytes_stats_invalid": false, 2026-02-20T18:01:22.142 INFO:tasks.workunit.client.0.trial195.stdout: "pin_stats_invalid": false, 2026-02-20T18:01:22.142 INFO:tasks.workunit.client.0.trial195.stdout: "manifest_stats_invalid": false, 2026-02-20T18:01:22.142 INFO:tasks.workunit.client.0.trial195.stdout: "snaptrimq_len": 0, 2026-02-20T18:01:22.142 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub_duration": 0, 2026-02-20T18:01:22.142 INFO:tasks.workunit.client.0.trial195.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-22T02:50:06.103041+0000", 2026-02-20T18:01:22.142 INFO:tasks.workunit.client.0.trial195.stdout: "scrub_duration": 0, 2026-02-20T18:01:22.142 INFO:tasks.workunit.client.0.trial195.stdout: "objects_trimmed": 0, 2026-02-20T18:01:22.142 INFO:tasks.workunit.client.0.trial195.stdout: "snaptrim_duration": 0, 2026-02-20T18:01:22.142 INFO:tasks.workunit.client.0.trial195.stdout: "stat_sum": { 2026-02-20T18:01:22.142 INFO:tasks.workunit.client.0.trial195.stdout: "num_bytes": 1232896, 2026-02-20T18:01:22.142 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects": 301, 2026-02-20T18:01:22.142 INFO:tasks.workunit.client.0.trial195.stdout: "num_object_clones": 0, 2026-02-20T18:01:22.142 INFO:tasks.workunit.client.0.trial195.stdout: "num_object_copies": 1505, 2026-02-20T18:01:22.142 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_missing_on_primary": 0, 2026-02-20T18:01:22.142 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_missing": 1, 2026-02-20T18:01:22.143 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_degraded": 301, 2026-02-20T18:01:22.143 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_misplaced": 0, 2026-02-20T18:01:22.143 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_unfound": 0, 2026-02-20T18:01:22.143 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_dirty": 301, 2026-02-20T18:01:22.143 INFO:tasks.workunit.client.0.trial195.stdout: "num_whiteouts": 0, 2026-02-20T18:01:22.143 INFO:tasks.workunit.client.0.trial195.stdout: "num_read": 0, 2026-02-20T18:01:22.143 INFO:tasks.workunit.client.0.trial195.stdout: "num_read_kb": 0, 2026-02-20T18:01:22.143 INFO:tasks.workunit.client.0.trial195.stdout: "num_write": 301, 2026-02-20T18:01:22.143 INFO:tasks.workunit.client.0.trial195.stdout: "num_write_kb": 1204, 2026-02-20T18:01:22.143 INFO:tasks.workunit.client.0.trial195.stdout: "num_scrub_errors": 0, 2026-02-20T18:01:22.143 INFO:tasks.workunit.client.0.trial195.stdout: "num_shallow_scrub_errors": 0, 2026-02-20T18:01:22.143 INFO:tasks.workunit.client.0.trial195.stdout: "num_deep_scrub_errors": 0, 2026-02-20T18:01:22.143 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_recovered": 0, 2026-02-20T18:01:22.143 INFO:tasks.workunit.client.0.trial195.stdout: "num_bytes_recovered": 0, 2026-02-20T18:01:22.144 INFO:tasks.workunit.client.0.trial195.stdout: "num_keys_recovered": 0, 2026-02-20T18:01:22.144 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_omap": 0, 2026-02-20T18:01:22.144 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_hit_set_archive": 0, 2026-02-20T18:01:22.144 INFO:tasks.workunit.client.0.trial195.stdout: "num_bytes_hit_set_archive": 0, 2026-02-20T18:01:22.144 INFO:tasks.workunit.client.0.trial195.stdout: "num_flush": 0, 2026-02-20T18:01:22.144 INFO:tasks.workunit.client.0.trial195.stdout: "num_flush_kb": 0, 2026-02-20T18:01:22.144 INFO:tasks.workunit.client.0.trial195.stdout: "num_evict": 0, 2026-02-20T18:01:22.144 INFO:tasks.workunit.client.0.trial195.stdout: "num_evict_kb": 0, 2026-02-20T18:01:22.144 INFO:tasks.workunit.client.0.trial195.stdout: "num_promote": 0, 2026-02-20T18:01:22.144 INFO:tasks.workunit.client.0.trial195.stdout: "num_flush_mode_high": 0, 2026-02-20T18:01:22.144 INFO:tasks.workunit.client.0.trial195.stdout: "num_flush_mode_low": 0, 2026-02-20T18:01:22.144 INFO:tasks.workunit.client.0.trial195.stdout: "num_evict_mode_some": 0, 2026-02-20T18:01:22.144 INFO:tasks.workunit.client.0.trial195.stdout: "num_evict_mode_full": 0, 2026-02-20T18:01:22.144 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_pinned": 0, 2026-02-20T18:01:22.144 INFO:tasks.workunit.client.0.trial195.stdout: "num_legacy_snapsets": 0, 2026-02-20T18:01:22.145 INFO:tasks.workunit.client.0.trial195.stdout: "num_large_omap_objects": 0, 2026-02-20T18:01:22.145 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_manifest": 0, 2026-02-20T18:01:22.145 INFO:tasks.workunit.client.0.trial195.stdout: "num_omap_bytes": 0, 2026-02-20T18:01:22.145 INFO:tasks.workunit.client.0.trial195.stdout: "num_omap_keys": 0, 2026-02-20T18:01:22.145 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_repaired": 0 2026-02-20T18:01:22.145 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:01:22.145 INFO:tasks.workunit.client.0.trial195.stdout: "up": [ 2026-02-20T18:01:22.145 INFO:tasks.workunit.client.0.trial195.stdout: 3, 2026-02-20T18:01:22.145 INFO:tasks.workunit.client.0.trial195.stdout: 1, 2026-02-20T18:01:22.145 INFO:tasks.workunit.client.0.trial195.stdout: 4, 2026-02-20T18:01:22.145 INFO:tasks.workunit.client.0.trial195.stdout: 0, 2026-02-20T18:01:22.145 INFO:tasks.workunit.client.0.trial195.stdout: 2147483647 2026-02-20T18:01:22.145 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:01:22.145 INFO:tasks.workunit.client.0.trial195.stdout: "acting": [ 2026-02-20T18:01:22.146 INFO:tasks.workunit.client.0.trial195.stdout: 3, 2026-02-20T18:01:22.146 INFO:tasks.workunit.client.0.trial195.stdout: 1, 2026-02-20T18:01:22.146 INFO:tasks.workunit.client.0.trial195.stdout: 4, 2026-02-20T18:01:22.146 INFO:tasks.workunit.client.0.trial195.stdout: 0, 2026-02-20T18:01:22.146 INFO:tasks.workunit.client.0.trial195.stdout: 2147483647 2026-02-20T18:01:22.146 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:01:22.146 INFO:tasks.workunit.client.0.trial195.stdout: "avail_no_missing": [ 2026-02-20T18:01:22.146 INFO:tasks.workunit.client.0.trial195.stdout: "3(0)", 2026-02-20T18:01:22.146 INFO:tasks.workunit.client.0.trial195.stdout: "0(3)", 2026-02-20T18:01:22.146 INFO:tasks.workunit.client.0.trial195.stdout: "1(1)", 2026-02-20T18:01:22.146 INFO:tasks.workunit.client.0.trial195.stdout: "4(2)" 2026-02-20T18:01:22.146 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:01:22.146 INFO:tasks.workunit.client.0.trial195.stdout: "object_location_counts": [ 2026-02-20T18:01:22.146 INFO:tasks.workunit.client.0.trial195.stdout: { 2026-02-20T18:01:22.146 INFO:tasks.workunit.client.0.trial195.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-20T18:01:22.146 INFO:tasks.workunit.client.0.trial195.stdout: "objects": 301 2026-02-20T18:01:22.147 INFO:tasks.workunit.client.0.trial195.stdout: } 2026-02-20T18:01:22.147 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:01:22.147 INFO:tasks.workunit.client.0.trial195.stdout: "blocked_by": [], 2026-02-20T18:01:22.147 INFO:tasks.workunit.client.0.trial195.stdout: "up_primary": 3, 2026-02-20T18:01:22.147 INFO:tasks.workunit.client.0.trial195.stdout: "acting_primary": 3, 2026-02-20T18:01:22.147 INFO:tasks.workunit.client.0.trial195.stdout: "purged_snaps": [] 2026-02-20T18:01:22.147 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:01:22.147 INFO:tasks.workunit.client.0.trial195.stdout: "empty": 0, 2026-02-20T18:01:22.147 INFO:tasks.workunit.client.0.trial195.stdout: "dne": 0, 2026-02-20T18:01:22.147 INFO:tasks.workunit.client.0.trial195.stdout: "incomplete": 0, 2026-02-20T18:01:22.147 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_started": 58, 2026-02-20T18:01:22.147 INFO:tasks.workunit.client.0.trial195.stdout: "hit_set_history": { 2026-02-20T18:01:22.147 INFO:tasks.workunit.client.0.trial195.stdout: "current_last_update": "0'0", 2026-02-20T18:01:22.147 INFO:tasks.workunit.client.0.trial195.stdout: "history": [] 2026-02-20T18:01:22.147 INFO:tasks.workunit.client.0.trial195.stdout: } 2026-02-20T18:01:22.147 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:01:22.148 INFO:tasks.workunit.client.0.trial195.stdout: { 2026-02-20T18:01:22.148 INFO:tasks.workunit.client.0.trial195.stdout: "peer": "4(2)", 2026-02-20T18:01:22.148 INFO:tasks.workunit.client.0.trial195.stdout: "pgid": "2.0s2", 2026-02-20T18:01:22.148 INFO:tasks.workunit.client.0.trial195.stdout: "shard": "2", 2026-02-20T18:01:22.148 INFO:tasks.workunit.client.0.trial195.stdout: "last_update": "47'301", 2026-02-20T18:01:22.148 INFO:tasks.workunit.client.0.trial195.stdout: "last_complete": "47'301", 2026-02-20T18:01:22.148 INFO:tasks.workunit.client.0.trial195.stdout: "log_tail": "47'200", 2026-02-20T18:01:22.148 INFO:tasks.workunit.client.0.trial195.stdout: "last_user_version": 301, 2026-02-20T18:01:22.148 INFO:tasks.workunit.client.0.trial195.stdout: "last_backfill": "MAX", 2026-02-20T18:01:22.148 INFO:tasks.workunit.client.0.trial195.stdout: "partial_writes_last_complete": [], 2026-02-20T18:01:22.148 INFO:tasks.workunit.client.0.trial195.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-20T18:01:22.148 INFO:tasks.workunit.client.0.trial195.stdout: "purged_snaps": [], 2026-02-20T18:01:22.148 INFO:tasks.workunit.client.0.trial195.stdout: "history": { 2026-02-20T18:01:22.148 INFO:tasks.workunit.client.0.trial195.stdout: "epoch_created": 41, 2026-02-20T18:01:22.148 INFO:tasks.workunit.client.0.trial195.stdout: "epoch_pool_created": 41, 2026-02-20T18:01:22.148 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_started": 58, 2026-02-20T18:01:22.149 INFO:tasks.workunit.client.0.trial195.stdout: "last_interval_started": 57, 2026-02-20T18:01:22.149 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_clean": 42, 2026-02-20T18:01:22.149 INFO:tasks.workunit.client.0.trial195.stdout: "last_interval_clean": 41, 2026-02-20T18:01:22.149 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_split": 0, 2026-02-20T18:01:22.149 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_marked_full": 0, 2026-02-20T18:01:22.149 INFO:tasks.workunit.client.0.trial195.stdout: "same_up_since": 57, 2026-02-20T18:01:22.149 INFO:tasks.workunit.client.0.trial195.stdout: "same_interval_since": 57, 2026-02-20T18:01:22.149 INFO:tasks.workunit.client.0.trial195.stdout: "same_primary_since": 41, 2026-02-20T18:01:22.149 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub": "0'0", 2026-02-20T18:01:22.149 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub_stamp": "2026-02-20T18:00:46.410154+0000", 2026-02-20T18:01:22.149 INFO:tasks.workunit.client.0.trial195.stdout: "last_deep_scrub": "0'0", 2026-02-20T18:01:22.149 INFO:tasks.workunit.client.0.trial195.stdout: "last_deep_scrub_stamp": "2026-02-20T18:00:46.410154+0000", 2026-02-20T18:01:22.149 INFO:tasks.workunit.client.0.trial195.stdout: "last_clean_scrub_stamp": "2026-02-20T18:00:46.410154+0000", 2026-02-20T18:01:22.149 INFO:tasks.workunit.client.0.trial195.stdout: "prior_readable_until_ub": 0 2026-02-20T18:01:22.149 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:01:22.149 INFO:tasks.workunit.client.0.trial195.stdout: "stats": { 2026-02-20T18:01:22.149 INFO:tasks.workunit.client.0.trial195.stdout: "version": "47'301", 2026-02-20T18:01:22.150 INFO:tasks.workunit.client.0.trial195.stdout: "reported_seq": 332, 2026-02-20T18:01:22.150 INFO:tasks.workunit.client.0.trial195.stdout: "reported_epoch": 47, 2026-02-20T18:01:22.150 INFO:tasks.workunit.client.0.trial195.stdout: "state": "active+undersized+degraded", 2026-02-20T18:01:22.150 INFO:tasks.workunit.client.0.trial195.stdout: "last_fresh": "2026-02-20T18:00:58.575617+0000", 2026-02-20T18:01:22.150 INFO:tasks.workunit.client.0.trial195.stdout: "last_change": "2026-02-20T18:00:52.178278+0000", 2026-02-20T18:01:22.150 INFO:tasks.workunit.client.0.trial195.stdout: "last_active": "2026-02-20T18:00:58.575617+0000", 2026-02-20T18:01:22.150 INFO:tasks.workunit.client.0.trial195.stdout: "last_peered": "2026-02-20T18:00:58.575617+0000", 2026-02-20T18:01:22.150 INFO:tasks.workunit.client.0.trial195.stdout: "last_clean": "2026-02-20T18:00:51.565581+0000", 2026-02-20T18:01:22.150 INFO:tasks.workunit.client.0.trial195.stdout: "last_became_active": "2026-02-20T18:00:52.178278+0000", 2026-02-20T18:01:22.150 INFO:tasks.workunit.client.0.trial195.stdout: "last_became_peered": "2026-02-20T18:00:52.178278+0000", 2026-02-20T18:01:22.150 INFO:tasks.workunit.client.0.trial195.stdout: "last_unstale": "2026-02-20T18:00:58.575617+0000", 2026-02-20T18:01:22.150 INFO:tasks.workunit.client.0.trial195.stdout: "last_undegraded": "2026-02-20T18:00:52.176952+0000", 2026-02-20T18:01:22.150 INFO:tasks.workunit.client.0.trial195.stdout: "last_fullsized": "2026-02-20T18:00:52.176862+0000", 2026-02-20T18:01:22.150 INFO:tasks.workunit.client.0.trial195.stdout: "mapping_epoch": 57, 2026-02-20T18:01:22.150 INFO:tasks.workunit.client.0.trial195.stdout: "log_start": "47'200", 2026-02-20T18:01:22.150 INFO:tasks.workunit.client.0.trial195.stdout: "ondisk_log_start": "47'200", 2026-02-20T18:01:22.151 INFO:tasks.workunit.client.0.trial195.stdout: "created": 41, 2026-02-20T18:01:22.151 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_clean": 42, 2026-02-20T18:01:22.151 INFO:tasks.workunit.client.0.trial195.stdout: "parent": "0.0", 2026-02-20T18:01:22.151 INFO:tasks.workunit.client.0.trial195.stdout: "parent_split_bits": 0, 2026-02-20T18:01:22.151 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub": "0'0", 2026-02-20T18:01:22.151 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub_stamp": "2026-02-20T18:00:46.410154+0000", 2026-02-20T18:01:22.151 INFO:tasks.workunit.client.0.trial195.stdout: "last_deep_scrub": "0'0", 2026-02-20T18:01:22.151 INFO:tasks.workunit.client.0.trial195.stdout: "last_deep_scrub_stamp": "2026-02-20T18:00:46.410154+0000", 2026-02-20T18:01:22.151 INFO:tasks.workunit.client.0.trial195.stdout: "last_clean_scrub_stamp": "2026-02-20T18:00:46.410154+0000", 2026-02-20T18:01:22.151 INFO:tasks.workunit.client.0.trial195.stdout: "objects_scrubbed": 0, 2026-02-20T18:01:22.151 INFO:tasks.workunit.client.0.trial195.stdout: "log_size": 101, 2026-02-20T18:01:22.151 INFO:tasks.workunit.client.0.trial195.stdout: "log_dups_size": 200, 2026-02-20T18:01:22.151 INFO:tasks.workunit.client.0.trial195.stdout: "ondisk_log_size": 101, 2026-02-20T18:01:22.151 INFO:tasks.workunit.client.0.trial195.stdout: "stats_invalid": false, 2026-02-20T18:01:22.151 INFO:tasks.workunit.client.0.trial195.stdout: "dirty_stats_invalid": false, 2026-02-20T18:01:22.151 INFO:tasks.workunit.client.0.trial195.stdout: "omap_stats_invalid": false, 2026-02-20T18:01:22.152 INFO:tasks.workunit.client.0.trial195.stdout: "hitset_stats_invalid": false, 2026-02-20T18:01:22.152 INFO:tasks.workunit.client.0.trial195.stdout: "hitset_bytes_stats_invalid": false, 2026-02-20T18:01:22.152 INFO:tasks.workunit.client.0.trial195.stdout: "pin_stats_invalid": false, 2026-02-20T18:01:22.152 INFO:tasks.workunit.client.0.trial195.stdout: "manifest_stats_invalid": false, 2026-02-20T18:01:22.152 INFO:tasks.workunit.client.0.trial195.stdout: "snaptrimq_len": 0, 2026-02-20T18:01:22.152 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub_duration": 0, 2026-02-20T18:01:22.152 INFO:tasks.workunit.client.0.trial195.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-22T02:50:06.103041+0000", 2026-02-20T18:01:22.152 INFO:tasks.workunit.client.0.trial195.stdout: "scrub_duration": 0, 2026-02-20T18:01:22.152 INFO:tasks.workunit.client.0.trial195.stdout: "objects_trimmed": 0, 2026-02-20T18:01:22.152 INFO:tasks.workunit.client.0.trial195.stdout: "snaptrim_duration": 0, 2026-02-20T18:01:22.152 INFO:tasks.workunit.client.0.trial195.stdout: "stat_sum": { 2026-02-20T18:01:22.152 INFO:tasks.workunit.client.0.trial195.stdout: "num_bytes": 1232896, 2026-02-20T18:01:22.152 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects": 301, 2026-02-20T18:01:22.152 INFO:tasks.workunit.client.0.trial195.stdout: "num_object_clones": 0, 2026-02-20T18:01:22.152 INFO:tasks.workunit.client.0.trial195.stdout: "num_object_copies": 1505, 2026-02-20T18:01:22.152 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_missing_on_primary": 0, 2026-02-20T18:01:22.152 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_missing": 0, 2026-02-20T18:01:22.152 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_degraded": 301, 2026-02-20T18:01:22.153 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_misplaced": 0, 2026-02-20T18:01:22.153 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_unfound": 0, 2026-02-20T18:01:22.153 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_dirty": 301, 2026-02-20T18:01:22.153 INFO:tasks.workunit.client.0.trial195.stdout: "num_whiteouts": 0, 2026-02-20T18:01:22.153 INFO:tasks.workunit.client.0.trial195.stdout: "num_read": 0, 2026-02-20T18:01:22.153 INFO:tasks.workunit.client.0.trial195.stdout: "num_read_kb": 0, 2026-02-20T18:01:22.153 INFO:tasks.workunit.client.0.trial195.stdout: "num_write": 301, 2026-02-20T18:01:22.153 INFO:tasks.workunit.client.0.trial195.stdout: "num_write_kb": 1204, 2026-02-20T18:01:22.153 INFO:tasks.workunit.client.0.trial195.stdout: "num_scrub_errors": 0, 2026-02-20T18:01:22.153 INFO:tasks.workunit.client.0.trial195.stdout: "num_shallow_scrub_errors": 0, 2026-02-20T18:01:22.153 INFO:tasks.workunit.client.0.trial195.stdout: "num_deep_scrub_errors": 0, 2026-02-20T18:01:22.153 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_recovered": 0, 2026-02-20T18:01:22.153 INFO:tasks.workunit.client.0.trial195.stdout: "num_bytes_recovered": 0, 2026-02-20T18:01:22.153 INFO:tasks.workunit.client.0.trial195.stdout: "num_keys_recovered": 0, 2026-02-20T18:01:22.153 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_omap": 0, 2026-02-20T18:01:22.153 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_hit_set_archive": 0, 2026-02-20T18:01:22.153 INFO:tasks.workunit.client.0.trial195.stdout: "num_bytes_hit_set_archive": 0, 2026-02-20T18:01:22.153 INFO:tasks.workunit.client.0.trial195.stdout: "num_flush": 0, 2026-02-20T18:01:22.154 INFO:tasks.workunit.client.0.trial195.stdout: "num_flush_kb": 0, 2026-02-20T18:01:22.154 INFO:tasks.workunit.client.0.trial195.stdout: "num_evict": 0, 2026-02-20T18:01:22.154 INFO:tasks.workunit.client.0.trial195.stdout: "num_evict_kb": 0, 2026-02-20T18:01:22.154 INFO:tasks.workunit.client.0.trial195.stdout: "num_promote": 0, 2026-02-20T18:01:22.154 INFO:tasks.workunit.client.0.trial195.stdout: "num_flush_mode_high": 0, 2026-02-20T18:01:22.154 INFO:tasks.workunit.client.0.trial195.stdout: "num_flush_mode_low": 0, 2026-02-20T18:01:22.154 INFO:tasks.workunit.client.0.trial195.stdout: "num_evict_mode_some": 0, 2026-02-20T18:01:22.154 INFO:tasks.workunit.client.0.trial195.stdout: "num_evict_mode_full": 0, 2026-02-20T18:01:22.154 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_pinned": 0, 2026-02-20T18:01:22.154 INFO:tasks.workunit.client.0.trial195.stdout: "num_legacy_snapsets": 0, 2026-02-20T18:01:22.154 INFO:tasks.workunit.client.0.trial195.stdout: "num_large_omap_objects": 0, 2026-02-20T18:01:22.154 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_manifest": 0, 2026-02-20T18:01:22.154 INFO:tasks.workunit.client.0.trial195.stdout: "num_omap_bytes": 0, 2026-02-20T18:01:22.154 INFO:tasks.workunit.client.0.trial195.stdout: "num_omap_keys": 0, 2026-02-20T18:01:22.154 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_repaired": 0 2026-02-20T18:01:22.154 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:01:22.154 INFO:tasks.workunit.client.0.trial195.stdout: "up": [ 2026-02-20T18:01:22.154 INFO:tasks.workunit.client.0.trial195.stdout: 3, 2026-02-20T18:01:22.155 INFO:tasks.workunit.client.0.trial195.stdout: 1, 2026-02-20T18:01:22.155 INFO:tasks.workunit.client.0.trial195.stdout: 4, 2026-02-20T18:01:22.155 INFO:tasks.workunit.client.0.trial195.stdout: 0, 2026-02-20T18:01:22.155 INFO:tasks.workunit.client.0.trial195.stdout: 2147483647 2026-02-20T18:01:22.155 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:01:22.155 INFO:tasks.workunit.client.0.trial195.stdout: "acting": [ 2026-02-20T18:01:22.155 INFO:tasks.workunit.client.0.trial195.stdout: 3, 2026-02-20T18:01:22.155 INFO:tasks.workunit.client.0.trial195.stdout: 1, 2026-02-20T18:01:22.155 INFO:tasks.workunit.client.0.trial195.stdout: 4, 2026-02-20T18:01:22.155 INFO:tasks.workunit.client.0.trial195.stdout: 0, 2026-02-20T18:01:22.155 INFO:tasks.workunit.client.0.trial195.stdout: 2147483647 2026-02-20T18:01:22.155 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:01:22.155 INFO:tasks.workunit.client.0.trial195.stdout: "avail_no_missing": [ 2026-02-20T18:01:22.155 INFO:tasks.workunit.client.0.trial195.stdout: "3(0)", 2026-02-20T18:01:22.155 INFO:tasks.workunit.client.0.trial195.stdout: "0(3)", 2026-02-20T18:01:22.155 INFO:tasks.workunit.client.0.trial195.stdout: "1(1)", 2026-02-20T18:01:22.155 INFO:tasks.workunit.client.0.trial195.stdout: "4(2)" 2026-02-20T18:01:22.155 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:01:22.155 INFO:tasks.workunit.client.0.trial195.stdout: "object_location_counts": [ 2026-02-20T18:01:22.156 INFO:tasks.workunit.client.0.trial195.stdout: { 2026-02-20T18:01:22.156 INFO:tasks.workunit.client.0.trial195.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-20T18:01:22.156 INFO:tasks.workunit.client.0.trial195.stdout: "objects": 301 2026-02-20T18:01:22.156 INFO:tasks.workunit.client.0.trial195.stdout: } 2026-02-20T18:01:22.156 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:01:22.156 INFO:tasks.workunit.client.0.trial195.stdout: "blocked_by": [], 2026-02-20T18:01:22.156 INFO:tasks.workunit.client.0.trial195.stdout: "up_primary": 3, 2026-02-20T18:01:22.156 INFO:tasks.workunit.client.0.trial195.stdout: "acting_primary": 3, 2026-02-20T18:01:22.156 INFO:tasks.workunit.client.0.trial195.stdout: "purged_snaps": [] 2026-02-20T18:01:22.156 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:01:22.156 INFO:tasks.workunit.client.0.trial195.stdout: "empty": 0, 2026-02-20T18:01:22.156 INFO:tasks.workunit.client.0.trial195.stdout: "dne": 0, 2026-02-20T18:01:22.156 INFO:tasks.workunit.client.0.trial195.stdout: "incomplete": 0, 2026-02-20T18:01:22.156 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_started": 58, 2026-02-20T18:01:22.156 INFO:tasks.workunit.client.0.trial195.stdout: "hit_set_history": { 2026-02-20T18:01:22.156 INFO:tasks.workunit.client.0.trial195.stdout: "current_last_update": "0'0", 2026-02-20T18:01:22.156 INFO:tasks.workunit.client.0.trial195.stdout: "history": [] 2026-02-20T18:01:22.156 INFO:tasks.workunit.client.0.trial195.stdout: } 2026-02-20T18:01:22.156 INFO:tasks.workunit.client.0.trial195.stdout: } 2026-02-20T18:01:22.157 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:01:22.157 INFO:tasks.workunit.client.0.trial195.stdout: "recovery_state": [ 2026-02-20T18:01:22.157 INFO:tasks.workunit.client.0.trial195.stdout: { 2026-02-20T18:01:22.157 INFO:tasks.workunit.client.0.trial195.stdout: "name": "Started/Primary/Active", 2026-02-20T18:01:22.157 INFO:tasks.workunit.client.0.trial195.stdout: "enter_time": "2026-02-20T18:01:16.804391+0000", 2026-02-20T18:01:22.157 INFO:tasks.workunit.client.0.trial195.stdout: "might_have_unfound": [ 2026-02-20T18:01:22.157 INFO:tasks.workunit.client.0.trial195.stdout: { 2026-02-20T18:01:22.157 INFO:tasks.workunit.client.0.trial195.stdout: "osd": "0(3)", 2026-02-20T18:01:22.157 INFO:tasks.workunit.client.0.trial195.stdout: "status": "already probed" 2026-02-20T18:01:22.157 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:01:22.157 INFO:tasks.workunit.client.0.trial195.stdout: { 2026-02-20T18:01:22.157 INFO:tasks.workunit.client.0.trial195.stdout: "osd": "1(1)", 2026-02-20T18:01:22.157 INFO:tasks.workunit.client.0.trial195.stdout: "status": "already probed" 2026-02-20T18:01:22.157 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:01:22.157 INFO:tasks.workunit.client.0.trial195.stdout: { 2026-02-20T18:01:22.157 INFO:tasks.workunit.client.0.trial195.stdout: "osd": "2(4)", 2026-02-20T18:01:22.157 INFO:tasks.workunit.client.0.trial195.stdout: "status": "osd is down" 2026-02-20T18:01:22.157 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:01:22.157 INFO:tasks.workunit.client.0.trial195.stdout: { 2026-02-20T18:01:22.158 INFO:tasks.workunit.client.0.trial195.stdout: "osd": "4(2)", 2026-02-20T18:01:22.158 INFO:tasks.workunit.client.0.trial195.stdout: "status": "already probed" 2026-02-20T18:01:22.158 INFO:tasks.workunit.client.0.trial195.stdout: } 2026-02-20T18:01:22.158 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:01:22.158 INFO:tasks.workunit.client.0.trial195.stdout: "recovery_progress": { 2026-02-20T18:01:22.158 INFO:tasks.workunit.client.0.trial195.stdout: "backfill_targets": [], 2026-02-20T18:01:22.158 INFO:tasks.workunit.client.0.trial195.stdout: "waiting_on_backfill": [], 2026-02-20T18:01:22.158 INFO:tasks.workunit.client.0.trial195.stdout: "last_backfill_started": "MIN", 2026-02-20T18:01:22.158 INFO:tasks.workunit.client.0.trial195.stdout: "backfill_info": { 2026-02-20T18:01:22.158 INFO:tasks.workunit.client.0.trial195.stdout: "begin": "MIN", 2026-02-20T18:01:22.158 INFO:tasks.workunit.client.0.trial195.stdout: "end": "MIN", 2026-02-20T18:01:22.158 INFO:tasks.workunit.client.0.trial195.stdout: "objects": [] 2026-02-20T18:01:22.158 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:01:22.158 INFO:tasks.workunit.client.0.trial195.stdout: "peer_backfill_info": [], 2026-02-20T18:01:22.158 INFO:tasks.workunit.client.0.trial195.stdout: "backfills_in_flight": [], 2026-02-20T18:01:22.158 INFO:tasks.workunit.client.0.trial195.stdout: "recovering": [], 2026-02-20T18:01:22.158 INFO:tasks.workunit.client.0.trial195.stdout: "pg_backend": { 2026-02-20T18:01:22.158 INFO:tasks.workunit.client.0.trial195.stdout: "recovery_ops": [], 2026-02-20T18:01:22.158 INFO:tasks.workunit.client.0.trial195.stdout: "read_ops": [] 2026-02-20T18:01:22.158 INFO:tasks.workunit.client.0.trial195.stdout: } 2026-02-20T18:01:22.158 INFO:tasks.workunit.client.0.trial195.stdout: } 2026-02-20T18:01:22.158 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:01:22.159 INFO:tasks.workunit.client.0.trial195.stdout: { 2026-02-20T18:01:22.159 INFO:tasks.workunit.client.0.trial195.stdout: "name": "Started", 2026-02-20T18:01:22.159 INFO:tasks.workunit.client.0.trial195.stdout: "enter_time": "2026-02-20T18:01:16.703589+0000" 2026-02-20T18:01:22.159 INFO:tasks.workunit.client.0.trial195.stdout: } 2026-02-20T18:01:22.159 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:01:22.159 INFO:tasks.workunit.client.0.trial195.stdout: "scrubber": { 2026-02-20T18:01:22.159 INFO:tasks.workunit.client.0.trial195.stdout: "active": false, 2026-02-20T18:01:22.159 INFO:tasks.workunit.client.0.trial195.stdout: "must_scrub": false, 2026-02-20T18:01:22.159 INFO:tasks.workunit.client.0.trial195.stdout: "must_deep_scrub": false, 2026-02-20T18:01:22.159 INFO:tasks.workunit.client.0.trial195.stdout: "must_repair": false, 2026-02-20T18:01:22.159 INFO:tasks.workunit.client.0.trial195.stdout: "scrub_reg_stamp": "2026-02-22T00:59:46.521401+0000", 2026-02-20T18:01:22.159 INFO:tasks.workunit.client.0.trial195.stdout: "schedule": "not registered for scrubbing" 2026-02-20T18:01:22.159 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:01:22.159 INFO:tasks.workunit.client.0.trial195.stdout: "agent_state": {} 2026-02-20T18:01:22.159 INFO:tasks.workunit.client.0.trial195.stdout:} 2026-02-20T18:01:22.160 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:590: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-20T18:01:22.160 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:590: TEST_ec_backfill_unfound: grep -q obj250 2026-02-20T18:01:22.183 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:592: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-20T18:01:22.183 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:592: TEST_ec_backfill_unfound: jq .available_might_have_unfound 2026-02-20T18:01:22.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:592: TEST_ec_backfill_unfound: check=true 2026-02-20T18:01:22.251 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: test true == true 2026-02-20T18:01:22.251 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:595: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-20T18:01:22.251 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:595: TEST_ec_backfill_unfound: jq '.might_have_unfound[0].status' 2026-02-20T18:01:22.318 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:595: TEST_ec_backfill_unfound: eval 'check="osd' is 'down"' 2026-02-20T18:01:22.318 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:595: TEST_ec_backfill_unfound: check='osd is down' 2026-02-20T18:01:22.318 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:596: TEST_ec_backfill_unfound: test 'osd is down' == 'osd is down' 2026-02-20T18:01:22.319 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-20T18:01:22.319 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: jq '.might_have_unfound[0].osd' 2026-02-20T18:01:22.388 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: eval 'check="2(4)"' 2026-02-20T18:01:22.388 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: check='2(4)' 2026-02-20T18:01:22.388 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:599: TEST_ec_backfill_unfound: test '2(4)' == '2(4)' 2026-02-20T18:01:22.388 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:601: TEST_ec_backfill_unfound: activate_osd td/test-erasure-eio 2 2026-02-20T18:01:22.388 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-20T18:01:22.388 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-20T18:01:22.388 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-20T18:01:22.388 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-20T18:01:22.388 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-20T18:01:22.388 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:01:22.388 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:01:22.388 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:01:22.388 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:01:22.388 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-20T18:01:22.388 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-20T18:01:22.388 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-20T18:01:22.388 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-20T18:01:22.388 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:01:22.388 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-20T18:01:22.388 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:01:22.388 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:01:22.388 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:01:22.389 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:01:22.389 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:01:22.389 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:01:22.389 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:01:22.389 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:01:22.389 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:01:22.389 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:01:22.389 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:01:22.389 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:01:22.389 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:01:22.389 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:01:22.389 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-20T18:01:22.389 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-20T18:01:22.389 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-20T18:01:22.389 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-20T18:01:22.389 INFO:tasks.workunit.client.0.trial195.stdout:start osd.2 2026-02-20T18:01:22.389 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:01:22.389 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-20T18:01:22.389 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-20T18:01:22.390 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-20T18:01:22.390 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-20T18:01:22.390 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-20T18:01:22.407 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:22.407+0000 7fbdd678a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:22.409 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:22.408+0000 7fbdd678a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:22.410 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:22.409+0000 7fbdd678a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:22.548 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:22.546+0000 7fbdd678a900 -1 Falling back to public interface 2026-02-20T18:01:22.624 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-20T18:01:22.624 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:01:22.624 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-20T18:01:22.625 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:01:22.625 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:01:22.625 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:01:22.625 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:01:22.625 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:01:22.626 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:01:22.626 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:01:22.850 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:01:22.850 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:22.849+0000 7fbdd678a900 -1 osd.2 56 log_to_monitors true 2026-02-20T18:01:23.851 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:01:23.851 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:01:23.851 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:01:23.851 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:01:23.851 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:01:23.851 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:01:24.074 INFO:tasks.workunit.client.0.trial195.stdout:osd.2 up in weight 1 up_from 60 up_thru 20 down_at 57 last_clean_interval [49,56) [v2:127.0.0.1:6818/688074748,v1:127.0.0.1:6819/688074748] [v2:127.0.0.1:6820/688074748,v1:127.0.0.1:6821/688074748] exists,up 6b55b97c-83c5-4cf0-8416-8dd1bfb81f42 2026-02-20T18:01:24.074 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:01:24.074 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:01:24.074 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:01:24.074 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:604: TEST_ec_backfill_unfound: timeout 5 rados -p pool-jerasure get obj250 td/test-erasure-eio/CHECK 2026-02-20T18:01:29.078 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:605: TEST_ec_backfill_unfound: test 124 = 124 2026-02-20T18:01:29.078 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:607: TEST_ec_backfill_unfound: ceph pg 2.0 mark_unfound_lost delete 2026-02-20T18:01:29.143 INFO:tasks.workunit.client.0.trial195.stderr:pg has no unfound objects 2026-02-20T18:01:29.149 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:609: TEST_ec_backfill_unfound: wait_for_clean 2026-02-20T18:01:29.149 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:01:29.149 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:01:29.149 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:01:29.149 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:01:29.150 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:01:29.150 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:01:29.150 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:01:29.150 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:01:29.150 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:01:29.232 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:01:29.232 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:01:29.232 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:01:29.232 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:01:29.232 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:01:29.232 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:01:29.458 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:01:29.458 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:01:29.458 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:01:29.458 INFO:tasks.workunit.client.0.trial195.stderr:3 2026-02-20T18:01:29.458 INFO:tasks.workunit.client.0.trial195.stderr:4' 2026-02-20T18:01:29.458 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:01:29.458 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:01:29.458 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:01:29.524 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803791 2026-02-20T18:01:29.524 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803791 2026-02-20T18:01:29.525 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803791' 2026-02-20T18:01:29.525 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:01:29.525 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:01:29.589 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607567 2026-02-20T18:01:29.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607567 2026-02-20T18:01:29.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803791 1-51539607567' 2026-02-20T18:01:29.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:01:29.590 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:01:29.652 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=257698037764 2026-02-20T18:01:29.653 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 257698037764 2026-02-20T18:01:29.653 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803791 1-51539607567 2-257698037764' 2026-02-20T18:01:29.653 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:01:29.653 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:01:29.718 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149710 2026-02-20T18:01:29.718 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149710 2026-02-20T18:01:29.719 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803791 1-51539607567 2-257698037764 3-111669149710' 2026-02-20T18:01:29.719 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:01:29.719 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-20T18:01:29.783 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855373 2026-02-20T18:01:29.783 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855373 2026-02-20T18:01:29.783 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803791 1-51539607567 2-257698037764 3-111669149710 4-150323855373' 2026-02-20T18:01:29.783 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:01:29.783 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803791 2026-02-20T18:01:29.783 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:01:29.784 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:01:29.784 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803791 2026-02-20T18:01:29.784 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:01:29.785 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803791 2026-02-20T18:01:29.786 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803791' 2026-02-20T18:01:29.786 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.0 seq 25769803791 2026-02-20T18:01:29.786 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:01:30.011 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803789 -lt 25769803791 2026-02-20T18:01:30.011 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:01:31.012 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:01:31.013 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:01:31.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803791 -lt 25769803791 2026-02-20T18:01:31.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:01:31.238 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607567 2026-02-20T18:01:31.238 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:01:31.238 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:01:31.239 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-51539607567 2026-02-20T18:01:31.239 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:01:31.240 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=51539607567 2026-02-20T18:01:31.240 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 51539607567' 2026-02-20T18:01:31.241 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.1 seq 51539607567 2026-02-20T18:01:31.241 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:01:31.464 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 51539607567 -lt 51539607567 2026-02-20T18:01:31.465 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:01:31.465 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-257698037764 2026-02-20T18:01:31.465 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:01:31.466 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:01:31.467 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-257698037764 2026-02-20T18:01:31.467 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:01:31.468 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=257698037764 2026-02-20T18:01:31.468 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.2 seq 257698037764 2026-02-20T18:01:31.468 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 257698037764' 2026-02-20T18:01:31.468 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:01:31.695 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 257698037764 -lt 257698037764 2026-02-20T18:01:31.695 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:01:31.696 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149710 2026-02-20T18:01:31.696 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:01:31.696 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:01:31.697 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149710 2026-02-20T18:01:31.697 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:01:31.698 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149710 2026-02-20T18:01:31.698 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149710' 2026-02-20T18:01:31.699 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.3 seq 111669149710 2026-02-20T18:01:31.699 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:01:31.922 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149710 -lt 111669149710 2026-02-20T18:01:31.922 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:01:31.923 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-150323855373 2026-02-20T18:01:31.923 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:01:31.923 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-20T18:01:31.924 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-150323855373 2026-02-20T18:01:31.924 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:01:31.925 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=150323855373 2026-02-20T18:01:31.925 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.4 seq 150323855373 2026-02-20T18:01:31.926 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 150323855373' 2026-02-20T18:01:31.926 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-20T18:01:32.153 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 150323855374 -lt 150323855373 2026-02-20T18:01:32.154 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:01:32.154 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:01:32.154 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:01:32.465 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:01:32.465 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:01:32.465 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:01:32.465 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:01:32.465 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:01:32.465 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:01:32.465 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:01:32.465 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:01:32.686 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:01:32.687 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:01:32.687 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:01:32.687 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:01:32.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:01:32.997 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:01:32.997 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:01:32.997 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: seq 1 300 2026-02-20T18:01:32.998 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:32.998 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj1 = obj250 ']' 2026-02-20T18:01:32.998 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj1 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.017 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.018 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.018 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj2 = obj250 ']' 2026-02-20T18:01:33.018 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj2 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.037 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.037 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj3 = obj250 ']' 2026-02-20T18:01:33.038 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj3 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.056 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.057 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.057 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj4 = obj250 ']' 2026-02-20T18:01:33.057 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj4 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.075 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.076 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.076 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj5 = obj250 ']' 2026-02-20T18:01:33.076 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj5 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.094 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.095 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.096 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj6 = obj250 ']' 2026-02-20T18:01:33.096 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj6 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.114 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.115 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.115 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj7 = obj250 ']' 2026-02-20T18:01:33.115 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj7 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.133 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.134 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.134 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj8 = obj250 ']' 2026-02-20T18:01:33.134 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj8 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.151 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.152 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.152 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj9 = obj250 ']' 2026-02-20T18:01:33.152 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj9 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.170 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.171 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.171 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj10 = obj250 ']' 2026-02-20T18:01:33.171 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj10 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.188 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.189 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.189 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj11 = obj250 ']' 2026-02-20T18:01:33.189 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj11 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.205 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.206 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.206 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj12 = obj250 ']' 2026-02-20T18:01:33.206 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj12 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.224 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.225 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.225 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj13 = obj250 ']' 2026-02-20T18:01:33.226 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj13 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.243 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.244 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.244 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj14 = obj250 ']' 2026-02-20T18:01:33.244 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj14 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.262 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.263 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.263 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj15 = obj250 ']' 2026-02-20T18:01:33.264 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj15 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.281 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.281 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.282 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj16 = obj250 ']' 2026-02-20T18:01:33.282 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj16 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.299 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.300 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.300 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj17 = obj250 ']' 2026-02-20T18:01:33.300 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj17 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.318 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.319 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.319 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj18 = obj250 ']' 2026-02-20T18:01:33.319 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj18 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.337 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.338 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.338 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj19 = obj250 ']' 2026-02-20T18:01:33.338 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj19 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.355 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.356 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.357 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj20 = obj250 ']' 2026-02-20T18:01:33.357 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj20 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.374 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.375 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.375 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj21 = obj250 ']' 2026-02-20T18:01:33.375 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj21 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.393 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.394 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.394 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj22 = obj250 ']' 2026-02-20T18:01:33.394 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj22 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.412 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.413 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.413 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj23 = obj250 ']' 2026-02-20T18:01:33.413 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj23 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.431 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.432 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.432 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj24 = obj250 ']' 2026-02-20T18:01:33.432 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj24 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.450 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.451 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.451 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj25 = obj250 ']' 2026-02-20T18:01:33.451 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj25 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.469 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.470 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.470 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj26 = obj250 ']' 2026-02-20T18:01:33.470 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj26 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.487 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.488 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.489 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj27 = obj250 ']' 2026-02-20T18:01:33.489 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj27 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.506 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.507 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.507 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj28 = obj250 ']' 2026-02-20T18:01:33.507 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj28 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.525 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.526 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.527 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj29 = obj250 ']' 2026-02-20T18:01:33.527 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj29 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.544 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.544 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.545 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj30 = obj250 ']' 2026-02-20T18:01:33.545 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj30 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.563 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.563 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.564 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj31 = obj250 ']' 2026-02-20T18:01:33.564 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj31 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.581 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.582 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.582 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj32 = obj250 ']' 2026-02-20T18:01:33.583 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj32 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.600 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.601 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.601 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj33 = obj250 ']' 2026-02-20T18:01:33.601 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj33 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.621 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.621 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj34 = obj250 ']' 2026-02-20T18:01:33.622 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj34 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.639 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.640 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.640 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj35 = obj250 ']' 2026-02-20T18:01:33.640 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj35 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.658 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.659 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.659 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj36 = obj250 ']' 2026-02-20T18:01:33.659 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj36 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.677 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.678 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.678 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj37 = obj250 ']' 2026-02-20T18:01:33.678 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj37 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.696 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.697 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.697 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj38 = obj250 ']' 2026-02-20T18:01:33.697 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj38 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.715 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.716 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.716 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj39 = obj250 ']' 2026-02-20T18:01:33.716 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj39 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.734 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.735 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.735 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj40 = obj250 ']' 2026-02-20T18:01:33.736 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj40 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.754 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.755 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.755 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj41 = obj250 ']' 2026-02-20T18:01:33.755 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj41 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.773 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.774 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.774 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj42 = obj250 ']' 2026-02-20T18:01:33.774 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj42 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.792 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.793 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.793 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj43 = obj250 ']' 2026-02-20T18:01:33.793 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj43 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.811 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.812 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.812 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj44 = obj250 ']' 2026-02-20T18:01:33.812 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj44 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.830 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.831 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.831 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj45 = obj250 ']' 2026-02-20T18:01:33.831 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj45 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.849 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.849 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj46 = obj250 ']' 2026-02-20T18:01:33.849 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj46 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.867 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.868 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.868 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj47 = obj250 ']' 2026-02-20T18:01:33.868 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj47 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.886 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.887 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.887 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj48 = obj250 ']' 2026-02-20T18:01:33.887 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj48 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.904 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.905 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.905 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj49 = obj250 ']' 2026-02-20T18:01:33.905 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj49 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.923 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.923 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.924 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj50 = obj250 ']' 2026-02-20T18:01:33.924 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj50 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.941 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.942 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.942 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj51 = obj250 ']' 2026-02-20T18:01:33.942 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj51 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.960 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.961 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.961 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj52 = obj250 ']' 2026-02-20T18:01:33.961 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj52 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.979 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:33.980 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:33.981 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj53 = obj250 ']' 2026-02-20T18:01:33.981 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj53 td/test-erasure-eio/CHECK 2026-02-20T18:01:33.998 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.000 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.000 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj54 = obj250 ']' 2026-02-20T18:01:34.000 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj54 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.017 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.019 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.019 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj55 = obj250 ']' 2026-02-20T18:01:34.019 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj55 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj56 = obj250 ']' 2026-02-20T18:01:34.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj56 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.054 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.055 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.055 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj57 = obj250 ']' 2026-02-20T18:01:34.055 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj57 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.073 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.074 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.074 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj58 = obj250 ']' 2026-02-20T18:01:34.074 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj58 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.092 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.093 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.094 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj59 = obj250 ']' 2026-02-20T18:01:34.094 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj59 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.111 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.112 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.112 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj60 = obj250 ']' 2026-02-20T18:01:34.113 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj60 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.130 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.131 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.131 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj61 = obj250 ']' 2026-02-20T18:01:34.131 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj61 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.149 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.150 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.151 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj62 = obj250 ']' 2026-02-20T18:01:34.151 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj62 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.168 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.169 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.169 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj63 = obj250 ']' 2026-02-20T18:01:34.169 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj63 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.187 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.188 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.188 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj64 = obj250 ']' 2026-02-20T18:01:34.188 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj64 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.207 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.208 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.208 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj65 = obj250 ']' 2026-02-20T18:01:34.208 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj65 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.225 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.226 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.226 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj66 = obj250 ']' 2026-02-20T18:01:34.226 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj66 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.244 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.245 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.245 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj67 = obj250 ']' 2026-02-20T18:01:34.245 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj67 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.264 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.265 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.265 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj68 = obj250 ']' 2026-02-20T18:01:34.265 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj68 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.283 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.284 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.284 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj69 = obj250 ']' 2026-02-20T18:01:34.284 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj69 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.302 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.303 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.303 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj70 = obj250 ']' 2026-02-20T18:01:34.303 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj70 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.321 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj71 = obj250 ']' 2026-02-20T18:01:34.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj71 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.340 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.341 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.341 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj72 = obj250 ']' 2026-02-20T18:01:34.341 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj72 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.359 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.360 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.360 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj73 = obj250 ']' 2026-02-20T18:01:34.360 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj73 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.378 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.379 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.379 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj74 = obj250 ']' 2026-02-20T18:01:34.379 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj74 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.396 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.397 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.397 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj75 = obj250 ']' 2026-02-20T18:01:34.397 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj75 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.415 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.416 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.416 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj76 = obj250 ']' 2026-02-20T18:01:34.416 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj76 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.434 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.435 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.436 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj77 = obj250 ']' 2026-02-20T18:01:34.436 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj77 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.453 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.455 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.455 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj78 = obj250 ']' 2026-02-20T18:01:34.455 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj78 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.473 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.473 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.473 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj79 = obj250 ']' 2026-02-20T18:01:34.474 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj79 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj80 = obj250 ']' 2026-02-20T18:01:34.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj80 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.510 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.511 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.511 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj81 = obj250 ']' 2026-02-20T18:01:34.511 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj81 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.530 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.531 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.531 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj82 = obj250 ']' 2026-02-20T18:01:34.531 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj82 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.548 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.549 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.549 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj83 = obj250 ']' 2026-02-20T18:01:34.549 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj83 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.568 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.569 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.569 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj84 = obj250 ']' 2026-02-20T18:01:34.569 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj84 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.586 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.587 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.587 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj85 = obj250 ']' 2026-02-20T18:01:34.588 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj85 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.606 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.607 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.607 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj86 = obj250 ']' 2026-02-20T18:01:34.607 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj86 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.626 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.627 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.627 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj87 = obj250 ']' 2026-02-20T18:01:34.627 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj87 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.645 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.646 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.647 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj88 = obj250 ']' 2026-02-20T18:01:34.647 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj88 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.665 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.666 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.666 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj89 = obj250 ']' 2026-02-20T18:01:34.666 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj89 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.683 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.684 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.684 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj90 = obj250 ']' 2026-02-20T18:01:34.684 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj90 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.702 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.703 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.703 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj91 = obj250 ']' 2026-02-20T18:01:34.703 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj91 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.720 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.722 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.722 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj92 = obj250 ']' 2026-02-20T18:01:34.722 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj92 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.740 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.741 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.741 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj93 = obj250 ']' 2026-02-20T18:01:34.741 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj93 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.758 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.759 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.759 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj94 = obj250 ']' 2026-02-20T18:01:34.759 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj94 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.776 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.777 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.778 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj95 = obj250 ']' 2026-02-20T18:01:34.778 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj95 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.795 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.796 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.796 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj96 = obj250 ']' 2026-02-20T18:01:34.796 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj96 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.814 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.815 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.815 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj97 = obj250 ']' 2026-02-20T18:01:34.815 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj97 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.833 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.834 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.834 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj98 = obj250 ']' 2026-02-20T18:01:34.834 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj98 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.852 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.852 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.853 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj99 = obj250 ']' 2026-02-20T18:01:34.853 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj99 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.870 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.871 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.871 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj100 = obj250 ']' 2026-02-20T18:01:34.871 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj100 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.889 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.890 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.890 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj101 = obj250 ']' 2026-02-20T18:01:34.890 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj101 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.908 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.909 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.909 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj102 = obj250 ']' 2026-02-20T18:01:34.909 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj102 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.928 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.929 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.929 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj103 = obj250 ']' 2026-02-20T18:01:34.929 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj103 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.947 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.948 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.948 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj104 = obj250 ']' 2026-02-20T18:01:34.948 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj104 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.965 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.966 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.966 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj105 = obj250 ']' 2026-02-20T18:01:34.966 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj105 td/test-erasure-eio/CHECK 2026-02-20T18:01:34.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:34.986 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:34.986 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj106 = obj250 ']' 2026-02-20T18:01:34.986 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj106 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.005 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.005 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj107 = obj250 ']' 2026-02-20T18:01:35.005 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj107 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.021 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.022 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.023 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj108 = obj250 ']' 2026-02-20T18:01:35.023 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj108 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.040 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.041 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.042 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj109 = obj250 ']' 2026-02-20T18:01:35.042 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj109 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.060 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.060 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.061 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj110 = obj250 ']' 2026-02-20T18:01:35.061 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj110 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj111 = obj250 ']' 2026-02-20T18:01:35.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj111 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.098 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.099 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.099 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj112 = obj250 ']' 2026-02-20T18:01:35.099 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj112 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.117 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.118 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.118 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj113 = obj250 ']' 2026-02-20T18:01:35.118 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj113 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.136 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.137 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.137 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj114 = obj250 ']' 2026-02-20T18:01:35.137 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj114 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.155 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.156 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.156 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj115 = obj250 ']' 2026-02-20T18:01:35.156 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj115 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.174 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.174 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.175 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj116 = obj250 ']' 2026-02-20T18:01:35.175 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj116 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.193 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.194 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.194 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj117 = obj250 ']' 2026-02-20T18:01:35.194 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj117 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.213 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.214 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.214 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj118 = obj250 ']' 2026-02-20T18:01:35.214 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj118 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.233 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj119 = obj250 ']' 2026-02-20T18:01:35.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj119 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.253 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.253 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj120 = obj250 ']' 2026-02-20T18:01:35.253 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj120 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.272 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.272 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.272 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj121 = obj250 ']' 2026-02-20T18:01:35.272 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj121 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.291 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.292 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.292 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj122 = obj250 ']' 2026-02-20T18:01:35.292 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj122 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.310 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.310 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj123 = obj250 ']' 2026-02-20T18:01:35.310 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj123 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.328 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.329 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.330 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj124 = obj250 ']' 2026-02-20T18:01:35.330 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj124 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.347 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.348 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.348 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj125 = obj250 ']' 2026-02-20T18:01:35.348 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj125 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.366 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.367 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.367 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj126 = obj250 ']' 2026-02-20T18:01:35.367 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj126 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.385 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.385 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.386 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj127 = obj250 ']' 2026-02-20T18:01:35.386 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj127 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.404 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.405 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.405 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj128 = obj250 ']' 2026-02-20T18:01:35.405 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj128 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.423 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.424 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.424 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj129 = obj250 ']' 2026-02-20T18:01:35.424 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj129 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.442 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.443 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.443 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj130 = obj250 ']' 2026-02-20T18:01:35.443 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj130 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.462 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.462 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.463 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj131 = obj250 ']' 2026-02-20T18:01:35.463 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj131 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.481 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.481 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.482 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj132 = obj250 ']' 2026-02-20T18:01:35.482 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj132 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.499 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.500 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.500 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj133 = obj250 ']' 2026-02-20T18:01:35.500 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj133 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.518 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.519 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.520 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj134 = obj250 ']' 2026-02-20T18:01:35.520 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj134 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.537 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.538 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.538 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj135 = obj250 ']' 2026-02-20T18:01:35.539 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj135 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.556 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.557 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.557 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj136 = obj250 ']' 2026-02-20T18:01:35.557 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj136 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.576 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.577 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.577 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj137 = obj250 ']' 2026-02-20T18:01:35.577 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj137 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.595 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.596 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.596 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj138 = obj250 ']' 2026-02-20T18:01:35.596 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj138 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.613 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.614 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.614 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj139 = obj250 ']' 2026-02-20T18:01:35.614 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj139 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.632 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.633 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.633 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj140 = obj250 ']' 2026-02-20T18:01:35.633 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj140 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.650 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.651 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.651 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj141 = obj250 ']' 2026-02-20T18:01:35.651 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj141 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.671 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.672 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.672 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj142 = obj250 ']' 2026-02-20T18:01:35.672 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj142 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.690 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.691 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.691 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj143 = obj250 ']' 2026-02-20T18:01:35.691 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj143 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.708 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.709 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.709 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj144 = obj250 ']' 2026-02-20T18:01:35.710 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj144 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.728 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.728 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.729 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj145 = obj250 ']' 2026-02-20T18:01:35.729 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj145 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.746 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.747 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.747 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj146 = obj250 ']' 2026-02-20T18:01:35.747 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj146 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.766 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.767 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.767 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj147 = obj250 ']' 2026-02-20T18:01:35.767 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj147 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.785 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.786 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.786 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj148 = obj250 ']' 2026-02-20T18:01:35.786 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj148 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.804 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.805 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.805 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj149 = obj250 ']' 2026-02-20T18:01:35.805 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj149 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.823 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.824 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.824 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj150 = obj250 ']' 2026-02-20T18:01:35.824 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj150 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.842 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.843 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.843 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj151 = obj250 ']' 2026-02-20T18:01:35.843 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj151 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.861 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.862 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.862 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj152 = obj250 ']' 2026-02-20T18:01:35.862 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj152 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.880 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.881 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.881 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj153 = obj250 ']' 2026-02-20T18:01:35.881 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj153 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.898 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.899 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.899 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj154 = obj250 ']' 2026-02-20T18:01:35.900 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj154 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.917 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.918 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.918 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj155 = obj250 ']' 2026-02-20T18:01:35.918 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj155 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.936 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.937 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.937 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj156 = obj250 ']' 2026-02-20T18:01:35.937 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj156 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.954 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.955 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.955 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj157 = obj250 ']' 2026-02-20T18:01:35.955 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj157 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.972 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.973 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj158 = obj250 ']' 2026-02-20T18:01:35.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj158 td/test-erasure-eio/CHECK 2026-02-20T18:01:35.991 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:35.992 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:35.992 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj159 = obj250 ']' 2026-02-20T18:01:35.992 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj159 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.010 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.011 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.011 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj160 = obj250 ']' 2026-02-20T18:01:36.012 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj160 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.029 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.030 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.031 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj161 = obj250 ']' 2026-02-20T18:01:36.031 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj161 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.049 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj162 = obj250 ']' 2026-02-20T18:01:36.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj162 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.068 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.069 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.069 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj163 = obj250 ']' 2026-02-20T18:01:36.069 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj163 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.087 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.088 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.088 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj164 = obj250 ']' 2026-02-20T18:01:36.088 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj164 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.106 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.107 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.107 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj165 = obj250 ']' 2026-02-20T18:01:36.108 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj165 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.125 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.126 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.126 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj166 = obj250 ']' 2026-02-20T18:01:36.126 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj166 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.146 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.146 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj167 = obj250 ']' 2026-02-20T18:01:36.146 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj167 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.164 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.165 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.165 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj168 = obj250 ']' 2026-02-20T18:01:36.166 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj168 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.184 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.185 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.185 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj169 = obj250 ']' 2026-02-20T18:01:36.185 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj169 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.203 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj170 = obj250 ']' 2026-02-20T18:01:36.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj170 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.222 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.223 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.223 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj171 = obj250 ']' 2026-02-20T18:01:36.223 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj171 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.241 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.242 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.242 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj172 = obj250 ']' 2026-02-20T18:01:36.242 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj172 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.260 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.261 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.261 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj173 = obj250 ']' 2026-02-20T18:01:36.261 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj173 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.279 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.280 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.280 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj174 = obj250 ']' 2026-02-20T18:01:36.281 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj174 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.299 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.300 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.300 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj175 = obj250 ']' 2026-02-20T18:01:36.300 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj175 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.318 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.319 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.319 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj176 = obj250 ']' 2026-02-20T18:01:36.319 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj176 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.337 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.338 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.338 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj177 = obj250 ']' 2026-02-20T18:01:36.339 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj177 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.357 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.358 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.358 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj178 = obj250 ']' 2026-02-20T18:01:36.358 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj178 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.376 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.377 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.377 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj179 = obj250 ']' 2026-02-20T18:01:36.377 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj179 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.396 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.397 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.397 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj180 = obj250 ']' 2026-02-20T18:01:36.397 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj180 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.416 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.417 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.417 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj181 = obj250 ']' 2026-02-20T18:01:36.418 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj181 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.436 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.437 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.437 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj182 = obj250 ']' 2026-02-20T18:01:36.437 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj182 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.455 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.456 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.456 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj183 = obj250 ']' 2026-02-20T18:01:36.457 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj183 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.474 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.475 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.475 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj184 = obj250 ']' 2026-02-20T18:01:36.475 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj184 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.493 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.494 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.495 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj185 = obj250 ']' 2026-02-20T18:01:36.495 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj185 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.512 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.513 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.513 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj186 = obj250 ']' 2026-02-20T18:01:36.514 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj186 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.532 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.533 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.533 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj187 = obj250 ']' 2026-02-20T18:01:36.533 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj187 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.551 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.552 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.553 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj188 = obj250 ']' 2026-02-20T18:01:36.553 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj188 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.570 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.571 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.571 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj189 = obj250 ']' 2026-02-20T18:01:36.571 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj189 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.591 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.591 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj190 = obj250 ']' 2026-02-20T18:01:36.591 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj190 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.609 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.610 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.610 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj191 = obj250 ']' 2026-02-20T18:01:36.610 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj191 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.629 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.629 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.630 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj192 = obj250 ']' 2026-02-20T18:01:36.630 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj192 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.647 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.648 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.648 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj193 = obj250 ']' 2026-02-20T18:01:36.648 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj193 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.666 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.667 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.667 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj194 = obj250 ']' 2026-02-20T18:01:36.667 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj194 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.686 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.687 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.687 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj195 = obj250 ']' 2026-02-20T18:01:36.688 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj195 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.705 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.705 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.706 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj196 = obj250 ']' 2026-02-20T18:01:36.706 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj196 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.723 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.724 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.724 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj197 = obj250 ']' 2026-02-20T18:01:36.724 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj197 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.742 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.743 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj198 = obj250 ']' 2026-02-20T18:01:36.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj198 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.762 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.763 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.764 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj199 = obj250 ']' 2026-02-20T18:01:36.764 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj199 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.781 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.782 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.782 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj200 = obj250 ']' 2026-02-20T18:01:36.782 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj200 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.800 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.801 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.801 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj201 = obj250 ']' 2026-02-20T18:01:36.801 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj201 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.820 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.820 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.821 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj202 = obj250 ']' 2026-02-20T18:01:36.821 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj202 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.839 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.839 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.840 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj203 = obj250 ']' 2026-02-20T18:01:36.840 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj203 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.857 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.858 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.858 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj204 = obj250 ']' 2026-02-20T18:01:36.858 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj204 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.877 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.877 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.877 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj205 = obj250 ']' 2026-02-20T18:01:36.878 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj205 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.896 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.897 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.897 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj206 = obj250 ']' 2026-02-20T18:01:36.897 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj206 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.915 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.916 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.916 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj207 = obj250 ']' 2026-02-20T18:01:36.916 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj207 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.934 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.935 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.935 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj208 = obj250 ']' 2026-02-20T18:01:36.935 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj208 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.953 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.954 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.954 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj209 = obj250 ']' 2026-02-20T18:01:36.954 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj209 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.972 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.972 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.973 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj210 = obj250 ']' 2026-02-20T18:01:36.973 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj210 td/test-erasure-eio/CHECK 2026-02-20T18:01:36.990 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:36.991 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:36.991 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj211 = obj250 ']' 2026-02-20T18:01:36.991 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj211 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.010 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.010 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.010 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj212 = obj250 ']' 2026-02-20T18:01:37.010 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj212 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.028 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.029 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.029 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj213 = obj250 ']' 2026-02-20T18:01:37.029 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj213 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.049 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj214 = obj250 ']' 2026-02-20T18:01:37.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj214 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.069 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.070 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.070 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj215 = obj250 ']' 2026-02-20T18:01:37.070 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj215 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.088 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.089 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.089 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj216 = obj250 ']' 2026-02-20T18:01:37.089 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj216 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.107 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.108 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.108 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj217 = obj250 ']' 2026-02-20T18:01:37.108 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj217 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.125 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.126 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.126 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj218 = obj250 ']' 2026-02-20T18:01:37.126 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj218 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.143 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.144 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.144 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj219 = obj250 ']' 2026-02-20T18:01:37.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj219 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.162 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.163 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.163 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj220 = obj250 ']' 2026-02-20T18:01:37.163 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj220 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.182 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.182 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.182 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj221 = obj250 ']' 2026-02-20T18:01:37.182 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj221 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.201 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.201 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.202 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj222 = obj250 ']' 2026-02-20T18:01:37.202 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj222 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.220 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.220 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.220 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj223 = obj250 ']' 2026-02-20T18:01:37.220 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj223 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.238 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.239 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.239 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj224 = obj250 ']' 2026-02-20T18:01:37.239 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj224 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.258 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.259 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.259 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj225 = obj250 ']' 2026-02-20T18:01:37.259 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj225 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.276 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj226 = obj250 ']' 2026-02-20T18:01:37.278 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj226 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.296 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.297 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.297 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj227 = obj250 ']' 2026-02-20T18:01:37.297 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj227 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.316 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.317 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.317 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj228 = obj250 ']' 2026-02-20T18:01:37.317 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj228 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.335 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.336 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.336 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj229 = obj250 ']' 2026-02-20T18:01:37.336 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj229 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.354 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.355 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.355 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj230 = obj250 ']' 2026-02-20T18:01:37.355 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj230 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.378 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.379 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.379 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj231 = obj250 ']' 2026-02-20T18:01:37.379 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj231 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.397 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.397 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.397 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj232 = obj250 ']' 2026-02-20T18:01:37.397 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj232 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.416 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.417 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.417 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj233 = obj250 ']' 2026-02-20T18:01:37.417 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj233 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.436 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.436 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.436 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj234 = obj250 ']' 2026-02-20T18:01:37.436 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj234 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.454 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.456 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.456 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj235 = obj250 ']' 2026-02-20T18:01:37.456 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj235 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.473 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.474 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.474 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj236 = obj250 ']' 2026-02-20T18:01:37.474 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj236 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.493 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.494 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.495 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj237 = obj250 ']' 2026-02-20T18:01:37.495 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj237 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.513 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.514 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.514 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj238 = obj250 ']' 2026-02-20T18:01:37.514 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj238 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.531 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.532 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.532 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj239 = obj250 ']' 2026-02-20T18:01:37.532 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj239 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.550 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.551 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.551 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj240 = obj250 ']' 2026-02-20T18:01:37.551 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj240 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.569 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.570 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.570 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj241 = obj250 ']' 2026-02-20T18:01:37.570 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj241 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.589 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.591 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.591 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj242 = obj250 ']' 2026-02-20T18:01:37.591 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj242 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.609 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.610 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.610 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj243 = obj250 ']' 2026-02-20T18:01:37.610 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj243 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.628 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.629 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.629 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj244 = obj250 ']' 2026-02-20T18:01:37.629 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj244 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.646 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.647 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.648 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj245 = obj250 ']' 2026-02-20T18:01:37.648 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj245 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.666 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.667 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.667 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj246 = obj250 ']' 2026-02-20T18:01:37.667 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj246 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.685 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.687 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.687 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj247 = obj250 ']' 2026-02-20T18:01:37.687 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj247 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.704 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.705 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.705 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj248 = obj250 ']' 2026-02-20T18:01:37.706 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj248 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.723 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.724 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.724 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj249 = obj250 ']' 2026-02-20T18:01:37.725 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj249 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.742 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.743 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj250 = obj250 ']' 2026-02-20T18:01:37.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:615: TEST_ec_backfill_unfound: rados -p pool-jerasure get obj250 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.760 INFO:tasks.workunit.client.0.trial195.stderr:error getting pool-jerasure/obj250: (2) No such file or directory 2026-02-20T18:01:37.761 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.761 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj251 = obj250 ']' 2026-02-20T18:01:37.761 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj251 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.778 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.779 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.779 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj252 = obj250 ']' 2026-02-20T18:01:37.779 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj252 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.797 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj253 = obj250 ']' 2026-02-20T18:01:37.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj253 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.815 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.816 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.816 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj254 = obj250 ']' 2026-02-20T18:01:37.817 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj254 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.835 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.836 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.836 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj255 = obj250 ']' 2026-02-20T18:01:37.836 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj255 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.854 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.855 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.855 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj256 = obj250 ']' 2026-02-20T18:01:37.855 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj256 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.873 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.874 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.875 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj257 = obj250 ']' 2026-02-20T18:01:37.875 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj257 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.893 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.894 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.894 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj258 = obj250 ']' 2026-02-20T18:01:37.895 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj258 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.912 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.913 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.913 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj259 = obj250 ']' 2026-02-20T18:01:37.914 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj259 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.931 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.932 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.932 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj260 = obj250 ']' 2026-02-20T18:01:37.932 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj260 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.950 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.951 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.951 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj261 = obj250 ']' 2026-02-20T18:01:37.951 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj261 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.970 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.971 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.971 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj262 = obj250 ']' 2026-02-20T18:01:37.971 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj262 td/test-erasure-eio/CHECK 2026-02-20T18:01:37.989 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:37.991 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:37.991 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj263 = obj250 ']' 2026-02-20T18:01:37.991 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj263 td/test-erasure-eio/CHECK 2026-02-20T18:01:38.009 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.010 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:38.010 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj264 = obj250 ']' 2026-02-20T18:01:38.010 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj264 td/test-erasure-eio/CHECK 2026-02-20T18:01:38.028 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.028 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:38.029 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj265 = obj250 ']' 2026-02-20T18:01:38.029 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj265 td/test-erasure-eio/CHECK 2026-02-20T18:01:38.048 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.049 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:38.049 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj266 = obj250 ']' 2026-02-20T18:01:38.049 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj266 td/test-erasure-eio/CHECK 2026-02-20T18:01:38.067 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.067 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:38.068 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj267 = obj250 ']' 2026-02-20T18:01:38.068 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj267 td/test-erasure-eio/CHECK 2026-02-20T18:01:38.086 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.087 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:38.087 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj268 = obj250 ']' 2026-02-20T18:01:38.087 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj268 td/test-erasure-eio/CHECK 2026-02-20T18:01:38.104 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.105 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:38.106 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj269 = obj250 ']' 2026-02-20T18:01:38.106 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj269 td/test-erasure-eio/CHECK 2026-02-20T18:01:38.123 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.124 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:38.124 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj270 = obj250 ']' 2026-02-20T18:01:38.124 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj270 td/test-erasure-eio/CHECK 2026-02-20T18:01:38.141 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.142 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:38.142 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj271 = obj250 ']' 2026-02-20T18:01:38.142 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj271 td/test-erasure-eio/CHECK 2026-02-20T18:01:38.161 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.162 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:38.162 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj272 = obj250 ']' 2026-02-20T18:01:38.162 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj272 td/test-erasure-eio/CHECK 2026-02-20T18:01:38.179 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.180 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:38.180 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj273 = obj250 ']' 2026-02-20T18:01:38.180 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj273 td/test-erasure-eio/CHECK 2026-02-20T18:01:38.198 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.199 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:38.199 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj274 = obj250 ']' 2026-02-20T18:01:38.199 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj274 td/test-erasure-eio/CHECK 2026-02-20T18:01:38.217 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.218 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:38.218 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj275 = obj250 ']' 2026-02-20T18:01:38.218 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj275 td/test-erasure-eio/CHECK 2026-02-20T18:01:38.236 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:38.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj276 = obj250 ']' 2026-02-20T18:01:38.238 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj276 td/test-erasure-eio/CHECK 2026-02-20T18:01:38.255 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.256 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:38.256 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj277 = obj250 ']' 2026-02-20T18:01:38.256 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj277 td/test-erasure-eio/CHECK 2026-02-20T18:01:38.274 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.275 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:38.275 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj278 = obj250 ']' 2026-02-20T18:01:38.275 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj278 td/test-erasure-eio/CHECK 2026-02-20T18:01:38.293 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.294 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:38.294 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj279 = obj250 ']' 2026-02-20T18:01:38.294 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj279 td/test-erasure-eio/CHECK 2026-02-20T18:01:38.312 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.312 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:38.313 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj280 = obj250 ']' 2026-02-20T18:01:38.313 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj280 td/test-erasure-eio/CHECK 2026-02-20T18:01:38.331 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.332 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:38.332 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj281 = obj250 ']' 2026-02-20T18:01:38.332 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj281 td/test-erasure-eio/CHECK 2026-02-20T18:01:38.349 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.350 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:38.350 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj282 = obj250 ']' 2026-02-20T18:01:38.350 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj282 td/test-erasure-eio/CHECK 2026-02-20T18:01:38.367 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.368 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:38.369 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj283 = obj250 ']' 2026-02-20T18:01:38.369 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj283 td/test-erasure-eio/CHECK 2026-02-20T18:01:38.386 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.387 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:38.387 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj284 = obj250 ']' 2026-02-20T18:01:38.387 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj284 td/test-erasure-eio/CHECK 2026-02-20T18:01:38.405 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.406 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:38.406 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj285 = obj250 ']' 2026-02-20T18:01:38.406 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj285 td/test-erasure-eio/CHECK 2026-02-20T18:01:38.423 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.424 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:38.425 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj286 = obj250 ']' 2026-02-20T18:01:38.425 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj286 td/test-erasure-eio/CHECK 2026-02-20T18:01:38.442 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.443 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:38.444 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj287 = obj250 ']' 2026-02-20T18:01:38.444 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj287 td/test-erasure-eio/CHECK 2026-02-20T18:01:38.461 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.462 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:38.462 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj288 = obj250 ']' 2026-02-20T18:01:38.462 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj288 td/test-erasure-eio/CHECK 2026-02-20T18:01:38.480 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.480 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:38.481 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj289 = obj250 ']' 2026-02-20T18:01:38.481 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj289 td/test-erasure-eio/CHECK 2026-02-20T18:01:38.499 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.500 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:38.501 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj290 = obj250 ']' 2026-02-20T18:01:38.501 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj290 td/test-erasure-eio/CHECK 2026-02-20T18:01:38.518 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.518 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:38.519 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj291 = obj250 ']' 2026-02-20T18:01:38.519 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj291 td/test-erasure-eio/CHECK 2026-02-20T18:01:38.536 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.537 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:38.537 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj292 = obj250 ']' 2026-02-20T18:01:38.537 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj292 td/test-erasure-eio/CHECK 2026-02-20T18:01:38.555 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.556 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:38.556 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj293 = obj250 ']' 2026-02-20T18:01:38.557 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj293 td/test-erasure-eio/CHECK 2026-02-20T18:01:38.574 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.575 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:38.575 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj294 = obj250 ']' 2026-02-20T18:01:38.575 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj294 td/test-erasure-eio/CHECK 2026-02-20T18:01:38.593 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.594 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:38.594 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj295 = obj250 ']' 2026-02-20T18:01:38.594 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj295 td/test-erasure-eio/CHECK 2026-02-20T18:01:38.612 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.613 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:38.614 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj296 = obj250 ']' 2026-02-20T18:01:38.614 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj296 td/test-erasure-eio/CHECK 2026-02-20T18:01:38.631 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.632 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:38.632 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj297 = obj250 ']' 2026-02-20T18:01:38.633 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj297 td/test-erasure-eio/CHECK 2026-02-20T18:01:38.650 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.651 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:38.651 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj298 = obj250 ']' 2026-02-20T18:01:38.651 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj298 td/test-erasure-eio/CHECK 2026-02-20T18:01:38.668 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:38.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj299 = obj250 ']' 2026-02-20T18:01:38.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj299 td/test-erasure-eio/CHECK 2026-02-20T18:01:38.687 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.688 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:01:38.688 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj300 = obj250 ']' 2026-02-20T18:01:38.688 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj300 td/test-erasure-eio/CHECK 2026-02-20T18:01:38.706 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.707 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:622: TEST_ec_backfill_unfound: rm -f td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:01:38.707 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:624: TEST_ec_backfill_unfound: delete_erasure_coded_pool pool-jerasure 2026-02-20T18:01:38.707 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-20T18:01:38.707 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-20T18:01:38.982 INFO:tasks.workunit.client.0.trial195.stderr:pool 'pool-jerasure' does not exist 2026-02-20T18:01:38.991 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-20T18:01:39.269 INFO:tasks.workunit.client.0.trial195.stderr:erasure-code-profile myprofile does not exist 2026-02-20T18:01:39.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-20T18:01:39.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-20T18:01:39.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T18:01:39.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-20T18:01:39.277 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:01:39.277 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:01:39.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:01:39.278 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:01:39.278 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:01:39.388 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:01:39.389 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T18:01:39.389 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T18:01:39.390 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T18:01:39.391 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T18:01:39.391 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T18:01:39.392 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T18:01:39.392 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:01:39.393 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T18:01:39.393 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T18:01:39.393 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:01:39.394 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T18:01:39.395 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T18:01:39.395 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-20T18:01:39.431 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T18:01:39.431 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:01:39.431 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:01:39.431 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.85808 2026-02-20T18:01:39.433 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T18:01:39.433 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T18:01:39.433 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-20T18:01:39.433 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-20T18:01:39.434 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-20T18:01:39.434 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-20T18:01:39.434 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-20T18:01:39.434 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T18:01:39.434 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-20T18:01:39.434 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:01:39.434 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:01:39.434 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:01:39.434 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:01:39.435 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:01:39.435 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:01:39.436 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T18:01:39.436 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T18:01:39.436 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T18:01:39.437 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T18:01:39.437 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T18:01:39.437 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T18:01:39.437 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:01:39.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T18:01:39.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T18:01:39.438 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:01:39.439 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T18:01:39.439 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T18:01:39.440 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-20T18:01:39.441 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T18:01:39.441 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:01:39.441 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:01:39.441 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.85808 2026-02-20T18:01:39.442 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T18:01:39.442 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T18:01:39.442 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-20T18:01:39.443 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-20T18:01:39.443 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:01:39.443 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:01:39.443 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.85808 2026-02-20T18:01:39.446 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-20T18:01:39.446 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-20T18:01:39.446 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-20T18:01:39.447 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-20T18:01:39.447 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-20T18:01:39.447 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-20T18:01:39.447 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-20T18:01:39.448 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-20T18:01:39.448 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-20T18:01:39.448 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-20T18:01:39.448 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-20T18:01:39.468 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-20T18:01:39.468 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:01:39.468 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:01:39.468 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:01:39.468 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:01:39.468 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:01:39.469 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:01:39.469 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-20T18:01:39.495 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-20T18:01:39.496 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-20T18:01:39.496 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T18:01:39.497 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T18:01:39.497 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-20T18:01:39.497 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-20T18:01:39.497 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T18:01:39.497 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:01:39.497 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:01:39.498 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:01:39.498 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:01:39.498 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:01:39.498 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:01:39.498 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T18:01:39.499 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.85808/ceph-mon.a.asok config get fsid 2026-02-20T18:01:39.545 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-20T18:01:39.545 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T18:01:39.545 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T18:01:39.545 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-20T18:01:39.545 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-20T18:01:39.545 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T18:01:39.545 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:01:39.545 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:01:39.546 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:01:39.546 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:01:39.546 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:01:39.546 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:01:39.546 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T18:01:39.546 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.85808/ceph-mon.a.asok config get mon_host 2026-02-20T18:01:39.589 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-20T18:01:39.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-20T18:01:39.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-20T18:01:39.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-20T18:01:39.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-20T18:01:39.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-20T18:01:39.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-20T18:01:39.692 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-20T18:01:39.692 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:01:39.693 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:01:39.693 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:01:39.693 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:01:39.693 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:01:39.693 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:01:39.693 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T18:01:39.694 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T18:01:39.710 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-20T18:01:39.710 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-20T18:01:39.807 INFO:tasks.workunit.client.0.trial195.stderr:pool 'rbd' already exists 2026-02-20T18:01:39.813 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-20T18:01:40.816 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-20T18:01:40.816 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:01:40.816 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:01:40.817 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:01:40.817 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:01:40.817 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:01:40.817 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:01:40.817 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-20T18:01:40.817 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-mon.a.asok log flush 2026-02-20T18:01:40.857 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T18:01:40.861 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-20T18:01:40.862 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T18:01:39.486+0000 7f7a22611d80 0 load: jerasure load: lrc 2026-02-20T18:01:40.863 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_object_attr_read_error td/test-erasure-eio 2026-02-20T18:01:40.863 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:379: TEST_ec_object_attr_read_error: local dir=td/test-erasure-eio 2026-02-20T18:01:40.863 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:380: TEST_ec_object_attr_read_error: local objname=myobject 2026-02-20T18:01:40.863 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:382: TEST_ec_object_attr_read_error: setup_osds 7 2026-02-20T18:01:40.864 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=7 2026-02-20T18:01:40.864 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-20T18:01:40.864 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 7 - 1 2026-02-20T18:01:40.869 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 6 2026-02-20T18:01:40.869 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:01:40.869 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-20T18:01:40.869 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:01:40.869 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:01:40.869 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-20T18:01:40.870 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:01:40.870 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-20T18:01:40.870 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:01:40.870 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:01:40.870 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:01:40.870 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:01:40.870 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-20T18:01:40.870 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-20T18:01:40.870 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:01:40.870 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:01:40.871 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:01:40.871 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:01:40.871 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:01:40.871 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:01:40.871 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:01:40.871 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:01:40.871 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:01:40.871 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:01:40.871 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:01:40.871 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:01:40.871 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:01:40.872 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:01:40.872 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:01:40.872 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:01:40.872 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:01:40.872 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:01:40.872 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:01:40.872 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:01:40.872 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:01:40.872 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:01:40.872 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-20T18:01:40.873 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:01:40.873 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=cbf30f62-25dc-400e-a9c1-926820613050 2026-02-20T18:01:40.873 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 cbf30f62-25dc-400e-a9c1-926820613050' 2026-02-20T18:01:40.873 INFO:tasks.workunit.client.0.trial195.stdout:add osd0 cbf30f62-25dc-400e-a9c1-926820613050 2026-02-20T18:01:40.873 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:01:40.882 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCEoZhp9kFxNBAA+CfFoJoXmtPC4fF9SQ6BXA== 2026-02-20T18:01:40.882 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCEoZhp9kFxNBAA+CfFoJoXmtPC4fF9SQ6BXA=="}' 2026-02-20T18:01:40.882 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new cbf30f62-25dc-400e-a9c1-926820613050 -i td/test-erasure-eio/0/new.json 2026-02-20T18:01:40.984 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:01:40.990 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-20T18:01:40.991 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCEoZhp9kFxNBAA+CfFoJoXmtPC4fF9SQ6BXA== --osd-uuid cbf30f62-25dc-400e-a9c1-926820613050 2026-02-20T18:01:41.010 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:41.008+0000 7f0510893900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:41.011 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:41.010+0000 7f0510893900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:41.013 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:41.011+0000 7f0510893900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:41.013 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:41.011+0000 7f0510893900 -1 bdev(0x5635f36d2c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-20T18:01:41.013 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:41.011+0000 7f0510893900 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-20T18:01:41.436 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-20T18:01:41.436 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:01:41.437 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-20T18:01:41.437 INFO:tasks.workunit.client.0.trial195.stdout:adding osd0 key to auth repository 2026-02-20T18:01:41.437 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:01:41.542 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-20T18:01:41.542 INFO:tasks.workunit.client.0.trial195.stdout:start osd.0 2026-02-20T18:01:41.544 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:01:41.544 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:01:41.544 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:01:41.544 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:01:41.559 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:41.558+0000 7f2aa621b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:41.561 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:41.559+0000 7f2aa621b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:41.562 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:41.560+0000 7f2aa621b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:41.642 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-20T18:01:41.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:01:41.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-20T18:01:41.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:01:41.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:01:41.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:01:41.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:01:41.643 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:01:41.644 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:01:41.644 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T18:01:41.740 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:41.737+0000 7f2aa621b900 -1 Falling back to public interface 2026-02-20T18:01:41.742 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:01:41.999 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:41.997+0000 7f2aa621b900 -1 osd.0 0 log_to_monitors true 2026-02-20T18:01:42.743 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:01:42.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:01:42.744 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:01:42.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:01:42.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:01:42.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T18:01:42.968 INFO:tasks.workunit.client.0.trial195.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1872826905,v1:127.0.0.1:6803/1872826905] [v2:127.0.0.1:6804/1872826905,v1:127.0.0.1:6805/1872826905] exists,up cbf30f62-25dc-400e-a9c1-926820613050 2026-02-20T18:01:42.972 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:01:42.972 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:01:42.972 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:01:42.973 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:01:42.973 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-20T18:01:42.973 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:01:42.973 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:01:42.973 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-20T18:01:42.973 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:01:42.973 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-20T18:01:42.973 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:01:42.973 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:01:42.973 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:01:42.973 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:01:42.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-20T18:01:42.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-20T18:01:42.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:01:42.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:01:42.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:01:42.974 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:01:42.974 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:01:42.974 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:01:42.974 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:01:42.974 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:01:42.974 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:01:42.974 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:01:42.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:01:42.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:01:42.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:01:42.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:01:42.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:01:42.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:01:42.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:01:42.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:01:42.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:01:42.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:01:42.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:01:42.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:01:42.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-20T18:01:42.976 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:01:42.976 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f7cd1b73-5795-406f-bdfe-9e4f2fb455ae 2026-02-20T18:01:42.976 INFO:tasks.workunit.client.0.trial195.stdout:add osd1 f7cd1b73-5795-406f-bdfe-9e4f2fb455ae 2026-02-20T18:01:42.976 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 f7cd1b73-5795-406f-bdfe-9e4f2fb455ae' 2026-02-20T18:01:42.977 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:01:42.984 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCGoZhpLhChOhAApGfQb2InWl0Mfn7H2Um8Nw== 2026-02-20T18:01:42.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCGoZhpLhChOhAApGfQb2InWl0Mfn7H2Um8Nw=="}' 2026-02-20T18:01:42.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f7cd1b73-5795-406f-bdfe-9e4f2fb455ae -i td/test-erasure-eio/1/new.json 2026-02-20T18:01:43.212 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:01:43.220 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-20T18:01:43.221 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCGoZhpLhChOhAApGfQb2InWl0Mfn7H2Um8Nw== --osd-uuid f7cd1b73-5795-406f-bdfe-9e4f2fb455ae 2026-02-20T18:01:43.240 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:43.238+0000 7f6c06e0f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:43.241 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:43.240+0000 7f6c06e0f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:43.242 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:43.241+0000 7f6c06e0f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:43.243 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:43.241+0000 7f6c06e0f900 -1 bdev(0x563214bb4c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-20T18:01:43.243 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:43.241+0000 7f6c06e0f900 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-20T18:01:43.555 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-20T18:01:43.556 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:01:43.556 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-20T18:01:43.556 INFO:tasks.workunit.client.0.trial195.stdout:adding osd1 key to auth repository 2026-02-20T18:01:43.556 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:01:43.891 INFO:tasks.workunit.client.0.trial195.stdout:start osd.1 2026-02-20T18:01:43.892 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-20T18:01:43.892 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:01:43.892 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:01:43.892 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:01:43.892 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:01:43.908 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:43.906+0000 7f3d0b023900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:43.909 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:43.908+0000 7f3d0b023900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:43.910 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:43.908+0000 7f3d0b023900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:44.052 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:44.050+0000 7f3d0b023900 -1 Falling back to public interface 2026-02-20T18:01:44.115 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-20T18:01:44.116 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:01:44.116 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-20T18:01:44.116 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:01:44.116 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:01:44.116 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:01:44.116 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:01:44.116 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:01:44.117 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:01:44.117 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:01:44.338 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:01:44.388 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:44.386+0000 7f3d0b023900 -1 osd.1 0 log_to_monitors true 2026-02-20T18:01:45.339 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:01:45.339 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:01:45.340 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:01:45.341 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:01:45.341 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:01:45.341 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:01:45.568 INFO:tasks.workunit.client.0.trial195.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/403203919,v1:127.0.0.1:6811/403203919] [v2:127.0.0.1:6812/403203919,v1:127.0.0.1:6813/403203919] exists,up f7cd1b73-5795-406f-bdfe-9e4f2fb455ae 2026-02-20T18:01:45.572 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:01:45.572 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:01:45.573 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:01:45.573 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:01:45.573 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-20T18:01:45.573 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:01:45.573 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:01:45.573 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-20T18:01:45.573 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:01:45.573 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-20T18:01:45.573 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:01:45.573 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:01:45.573 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:01:45.573 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:01:45.574 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-20T18:01:45.574 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-20T18:01:45.574 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:01:45.574 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:01:45.574 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:01:45.574 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:01:45.574 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:01:45.574 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:01:45.574 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:01:45.575 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:01:45.575 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:01:45.575 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:01:45.575 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:01:45.575 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:01:45.575 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:01:45.575 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:01:45.576 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:01:45.576 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:01:45.576 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:01:45.576 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:01:45.576 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:01:45.576 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:01:45.576 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:01:45.577 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:01:45.577 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-20T18:01:45.577 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:01:45.577 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c185ea25-da73-4557-8b35-2a82046e7ac6 2026-02-20T18:01:45.577 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 c185ea25-da73-4557-8b35-2a82046e7ac6' 2026-02-20T18:01:45.578 INFO:tasks.workunit.client.0.trial195.stdout:add osd2 c185ea25-da73-4557-8b35-2a82046e7ac6 2026-02-20T18:01:45.578 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:01:45.584 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCJoZhp00jBIhAAdA/MiUMuQ7mY6UFzbTh3vw== 2026-02-20T18:01:45.584 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCJoZhp00jBIhAAdA/MiUMuQ7mY6UFzbTh3vw=="}' 2026-02-20T18:01:45.584 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c185ea25-da73-4557-8b35-2a82046e7ac6 -i td/test-erasure-eio/2/new.json 2026-02-20T18:01:45.813 INFO:tasks.workunit.client.0.trial195.stdout:2 2026-02-20T18:01:45.822 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-20T18:01:45.823 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCJoZhp00jBIhAAdA/MiUMuQ7mY6UFzbTh3vw== --osd-uuid c185ea25-da73-4557-8b35-2a82046e7ac6 2026-02-20T18:01:45.841 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:45.839+0000 7fa844697900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:45.842 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:45.841+0000 7fa844697900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:45.843 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:45.842+0000 7fa844697900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:45.844 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:45.842+0000 7fa844697900 -1 bdev(0x5592166fcc00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-20T18:01:45.844 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:45.842+0000 7fa844697900 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-20T18:01:46.157 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-20T18:01:46.158 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:01:46.158 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-20T18:01:46.159 INFO:tasks.workunit.client.0.trial195.stdout:adding osd2 key to auth repository 2026-02-20T18:01:46.159 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:01:46.478 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-20T18:01:46.478 INFO:tasks.workunit.client.0.trial195.stdout:start osd.2 2026-02-20T18:01:46.478 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:01:46.478 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:01:46.478 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:01:46.479 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:01:46.496 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:46.495+0000 7fcb6f579900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:46.498 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:46.497+0000 7fcb6f579900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:46.498 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:46.498+0000 7fcb6f579900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:46.620 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:46.619+0000 7fcb6f579900 -1 Falling back to public interface 2026-02-20T18:01:46.704 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-20T18:01:46.705 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:01:46.705 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-20T18:01:46.705 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:01:46.705 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:01:46.705 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:01:46.706 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:01:46.706 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:01:46.706 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:01:46.706 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:01:46.901 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:46.900+0000 7fcb6f579900 -1 osd.2 0 log_to_monitors true 2026-02-20T18:01:46.931 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:01:47.932 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:01:47.932 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:01:47.933 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:01:47.933 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:01:47.933 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:01:47.933 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:01:48.163 INFO:tasks.workunit.client.0.trial195.stdout:osd.2 up in weight 1 up_from 18 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3038260300,v1:127.0.0.1:6819/3038260300] [v2:127.0.0.1:6820/3038260300,v1:127.0.0.1:6821/3038260300] exists,up c185ea25-da73-4557-8b35-2a82046e7ac6 2026-02-20T18:01:48.167 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:01:48.167 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:01:48.167 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:01:48.167 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:01:48.168 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-20T18:01:48.168 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:01:48.168 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:01:48.168 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-20T18:01:48.168 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:01:48.168 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-20T18:01:48.168 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:01:48.168 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:01:48.168 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:01:48.168 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:01:48.169 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-20T18:01:48.169 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-20T18:01:48.169 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:01:48.169 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:01:48.169 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:01:48.169 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:01:48.169 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:01:48.169 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:01:48.169 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:01:48.169 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:01:48.169 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:01:48.170 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:01:48.170 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:01:48.170 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:01:48.170 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:01:48.170 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:01:48.170 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:01:48.170 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:01:48.170 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:01:48.170 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:01:48.170 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:01:48.170 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:01:48.171 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:01:48.171 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:01:48.171 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-20T18:01:48.171 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:01:48.171 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=3dd0db4b-4be8-4deb-8812-b25061fc9d5b 2026-02-20T18:01:48.171 INFO:tasks.workunit.client.0.trial195.stdout:add osd3 3dd0db4b-4be8-4deb-8812-b25061fc9d5b 2026-02-20T18:01:48.172 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 3dd0db4b-4be8-4deb-8812-b25061fc9d5b' 2026-02-20T18:01:48.172 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:01:48.179 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCMoZhpe92cChAAqa3x9HR4dkBVVx9xnWHL3g== 2026-02-20T18:01:48.179 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCMoZhpe92cChAAqa3x9HR4dkBVVx9xnWHL3g=="}' 2026-02-20T18:01:48.179 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 3dd0db4b-4be8-4deb-8812-b25061fc9d5b -i td/test-erasure-eio/3/new.json 2026-02-20T18:01:48.406 INFO:tasks.workunit.client.0.trial195.stdout:3 2026-02-20T18:01:48.415 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-20T18:01:48.416 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCMoZhpe92cChAAqa3x9HR4dkBVVx9xnWHL3g== --osd-uuid 3dd0db4b-4be8-4deb-8812-b25061fc9d5b 2026-02-20T18:01:48.435 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:48.434+0000 7f87bf80e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:48.437 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:48.436+0000 7f87bf80e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:48.438 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:48.437+0000 7f87bf80e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:48.438 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:48.437+0000 7f87bf80e900 -1 bdev(0x5578e17c4c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-20T18:01:48.438 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:48.437+0000 7f87bf80e900 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-20T18:01:48.763 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-20T18:01:48.763 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:01:48.764 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-20T18:01:48.764 INFO:tasks.workunit.client.0.trial195.stdout:adding osd3 key to auth repository 2026-02-20T18:01:48.764 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:01:49.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-20T18:01:49.081 INFO:tasks.workunit.client.0.trial195.stdout:start osd.3 2026-02-20T18:01:49.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:01:49.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:01:49.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:01:49.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:01:49.098 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:49.098+0000 7f225f223900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:49.100 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:49.099+0000 7f225f223900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:49.101 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:49.100+0000 7f225f223900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:49.225 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:49.224+0000 7f225f223900 -1 Falling back to public interface 2026-02-20T18:01:49.306 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-20T18:01:49.306 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:01:49.306 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-20T18:01:49.306 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:01:49.306 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:01:49.306 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:01:49.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:01:49.307 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:01:49.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:01:49.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T18:01:49.529 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:01:49.539 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:49.538+0000 7f225f223900 -1 osd.3 0 log_to_monitors true 2026-02-20T18:01:50.530 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:01:50.530 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:01:50.530 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:01:50.531 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:01:50.531 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:01:50.531 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T18:01:50.754 INFO:tasks.workunit.client.0.trial195.stdout:osd.3 up in weight 1 up_from 27 up_thru 28 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1365826794,v1:127.0.0.1:6827/1365826794] [v2:127.0.0.1:6828/1365826794,v1:127.0.0.1:6829/1365826794] exists,up 3dd0db4b-4be8-4deb-8812-b25061fc9d5b 2026-02-20T18:01:50.755 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:01:50.755 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:01:50.755 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:01:50.755 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:01:50.755 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-20T18:01:50.755 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:01:50.755 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:01:50.755 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-20T18:01:50.755 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:01:50.755 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-20T18:01:50.755 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:01:50.755 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:01:50.755 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:01:50.755 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:01:50.755 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-20T18:01:50.755 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-20T18:01:50.755 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:01:50.755 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:01:50.755 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:01:50.755 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:01:50.755 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:01:50.755 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:01:50.756 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:01:50.756 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:01:50.756 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:01:50.756 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:01:50.756 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:01:50.756 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:01:50.756 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:01:50.756 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:01:50.756 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:01:50.757 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:01:50.757 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:01:50.757 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:01:50.757 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:01:50.757 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:01:50.757 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:01:50.757 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:01:50.757 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-20T18:01:50.757 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:01:50.757 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=9e053c1c-b2de-4b0a-8a41-45df935b89eb 2026-02-20T18:01:50.757 INFO:tasks.workunit.client.0.trial195.stdout:add osd4 9e053c1c-b2de-4b0a-8a41-45df935b89eb 2026-02-20T18:01:50.758 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 9e053c1c-b2de-4b0a-8a41-45df935b89eb' 2026-02-20T18:01:50.758 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:01:50.769 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCOoZhpHCPSLRAAgOAwv3xuXm6m92P4oB+pzg== 2026-02-20T18:01:50.769 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCOoZhpHCPSLRAAgOAwv3xuXm6m92P4oB+pzg=="}' 2026-02-20T18:01:50.769 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 9e053c1c-b2de-4b0a-8a41-45df935b89eb -i td/test-erasure-eio/4/new.json 2026-02-20T18:01:51.000 INFO:tasks.workunit.client.0.trial195.stdout:4 2026-02-20T18:01:51.008 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-20T18:01:51.009 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCOoZhpHCPSLRAAgOAwv3xuXm6m92P4oB+pzg== --osd-uuid 9e053c1c-b2de-4b0a-8a41-45df935b89eb 2026-02-20T18:01:51.027 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:51.026+0000 7fed1a346900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:51.029 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:51.028+0000 7fed1a346900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:51.030 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:51.029+0000 7fed1a346900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:51.030 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:51.029+0000 7fed1a346900 -1 bdev(0x55af9cc08c00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-20T18:01:51.030 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:51.029+0000 7fed1a346900 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-20T18:01:51.342 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-20T18:01:51.342 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:01:51.343 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-20T18:01:51.343 INFO:tasks.workunit.client.0.trial195.stdout:adding osd4 key to auth repository 2026-02-20T18:01:51.344 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:01:51.656 INFO:tasks.workunit.client.0.trial195.stdout:start osd.4 2026-02-20T18:01:51.657 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-20T18:01:51.657 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:01:51.657 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:01:51.657 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:01:51.657 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:01:51.675 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:51.674+0000 7f1621010900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:51.676 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:51.676+0000 7f1621010900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:51.677 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:51.676+0000 7f1621010900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:51.800 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:51.799+0000 7f1621010900 -1 Falling back to public interface 2026-02-20T18:01:51.882 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-20T18:01:51.882 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:01:51.882 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-20T18:01:51.882 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:01:51.882 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:01:51.882 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:01:51.882 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:01:51.882 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:01:51.882 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:01:51.882 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-20T18:01:52.108 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:01:52.137 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:52.136+0000 7f1621010900 -1 osd.4 0 log_to_monitors true 2026-02-20T18:01:53.109 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:01:53.109 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:01:53.109 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:01:53.110 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:01:53.110 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:01:53.110 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-20T18:01:53.340 INFO:tasks.workunit.client.0.trial195.stdout:osd.4 up in weight 1 up_from 35 up_thru 36 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/497469397,v1:127.0.0.1:6835/497469397] [v2:127.0.0.1:6836/497469397,v1:127.0.0.1:6837/497469397] exists,up 9e053c1c-b2de-4b0a-8a41-45df935b89eb 2026-02-20T18:01:53.345 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:01:53.345 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:01:53.345 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:01:53.345 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:01:53.345 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-20T18:01:53.345 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:01:53.345 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:01:53.346 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-20T18:01:53.346 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:01:53.346 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-20T18:01:53.346 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:01:53.346 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:01:53.346 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:01:53.346 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:01:53.346 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-20T18:01:53.347 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-20T18:01:53.347 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:01:53.347 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:01:53.347 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:01:53.347 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:01:53.348 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:01:53.348 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:01:53.348 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:01:53.348 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:01:53.348 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:01:53.348 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:01:53.348 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:01:53.349 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:01:53.349 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:01:53.349 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:01:53.349 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:01:53.349 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:01:53.349 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:01:53.349 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:01:53.349 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:01:53.349 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:01:53.350 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:01:53.350 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:01:53.350 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-20T18:01:53.350 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:01:53.350 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=760b4938-fc7d-46dd-82be-01f37a79973d 2026-02-20T18:01:53.350 INFO:tasks.workunit.client.0.trial195.stdout:add osd5 760b4938-fc7d-46dd-82be-01f37a79973d 2026-02-20T18:01:53.351 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 760b4938-fc7d-46dd-82be-01f37a79973d' 2026-02-20T18:01:53.351 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:01:53.356 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCRoZhpPsUiFRAARizbPvf5j+3yUZHPWXfY+w== 2026-02-20T18:01:53.356 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCRoZhpPsUiFRAARizbPvf5j+3yUZHPWXfY+w=="}' 2026-02-20T18:01:53.356 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 760b4938-fc7d-46dd-82be-01f37a79973d -i td/test-erasure-eio/5/new.json 2026-02-20T18:01:53.584 INFO:tasks.workunit.client.0.trial195.stdout:5 2026-02-20T18:01:53.591 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-20T18:01:53.592 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCRoZhpPsUiFRAARizbPvf5j+3yUZHPWXfY+w== --osd-uuid 760b4938-fc7d-46dd-82be-01f37a79973d 2026-02-20T18:01:53.612 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:53.611+0000 7f7d2a2d9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:53.613 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:53.613+0000 7f7d2a2d9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:53.614 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:53.614+0000 7f7d2a2d9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:53.615 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:53.614+0000 7f7d2a2d9900 -1 bdev(0x559031d28c00 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-20T18:01:53.615 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:53.614+0000 7f7d2a2d9900 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-20T18:01:53.950 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-20T18:01:53.950 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:01:53.951 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-20T18:01:53.951 INFO:tasks.workunit.client.0.trial195.stdout:adding osd5 key to auth repository 2026-02-20T18:01:53.951 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:01:54.267 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-20T18:01:54.267 INFO:tasks.workunit.client.0.trial195.stdout:start osd.5 2026-02-20T18:01:54.268 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:01:54.268 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:01:54.268 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:01:54.268 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:01:54.284 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:54.283+0000 7f0affb03900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:54.285 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:54.284+0000 7f0affb03900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:54.286 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:54.285+0000 7f0affb03900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:54.454 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:54.452+0000 7f0affb03900 -1 Falling back to public interface 2026-02-20T18:01:54.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-20T18:01:54.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:01:54.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-20T18:01:54.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:01:54.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:01:54.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:01:54.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:01:54.492 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:01:54.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:01:54.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-20T18:01:54.717 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:01:54.753 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:54.752+0000 7f0affb03900 -1 osd.5 0 log_to_monitors true 2026-02-20T18:01:55.718 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:01:55.718 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:01:55.719 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:01:55.719 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:01:55.719 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:01:55.719 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-20T18:01:55.950 INFO:tasks.workunit.client.0.trial195.stdout:osd.5 up in weight 1 up_from 44 up_thru 45 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/3367210630,v1:127.0.0.1:6843/3367210630] [v2:127.0.0.1:6844/3367210630,v1:127.0.0.1:6845/3367210630] exists,up 760b4938-fc7d-46dd-82be-01f37a79973d 2026-02-20T18:01:55.955 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:01:55.956 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:01:55.956 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:01:55.956 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:01:55.956 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-20T18:01:55.956 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:01:55.956 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:01:55.956 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-20T18:01:55.957 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:01:55.957 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-20T18:01:55.957 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:01:55.957 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:01:55.958 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:01:55.958 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:01:55.958 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-20T18:01:55.958 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-20T18:01:55.958 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:01:55.959 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:01:55.959 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:01:55.959 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:01:55.959 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:01:55.959 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:01:55.959 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:01:55.960 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:01:55.960 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:01:55.960 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:01:55.960 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:01:55.960 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:01:55.960 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:01:55.960 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:01:55.960 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:01:55.960 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:01:55.960 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:01:55.961 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:01:55.961 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:01:55.961 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:01:55.961 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:01:55.961 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:01:55.962 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-20T18:01:55.962 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:01:55.962 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c316bd4f-9938-4243-aa58-b14c61dedbbc 2026-02-20T18:01:55.963 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 c316bd4f-9938-4243-aa58-b14c61dedbbc' 2026-02-20T18:01:55.963 INFO:tasks.workunit.client.0.trial195.stdout:add osd6 c316bd4f-9938-4243-aa58-b14c61dedbbc 2026-02-20T18:01:55.964 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:01:55.966 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCToZhpLI2CORAATEWRTT6pONQBjQgQlyhLpg== 2026-02-20T18:01:55.966 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCToZhpLI2CORAATEWRTT6pONQBjQgQlyhLpg=="}' 2026-02-20T18:01:55.966 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c316bd4f-9938-4243-aa58-b14c61dedbbc -i td/test-erasure-eio/6/new.json 2026-02-20T18:01:56.192 INFO:tasks.workunit.client.0.trial195.stdout:6 2026-02-20T18:01:56.201 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-20T18:01:56.202 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCToZhpLI2CORAATEWRTT6pONQBjQgQlyhLpg== --osd-uuid c316bd4f-9938-4243-aa58-b14c61dedbbc 2026-02-20T18:01:56.220 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:56.219+0000 7f9789d05900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:56.221 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:56.220+0000 7f9789d05900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:56.222 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:56.221+0000 7f9789d05900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:56.223 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:56.222+0000 7f9789d05900 -1 bdev(0x55c8b7ea6c00 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-20T18:01:56.223 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:56.222+0000 7f9789d05900 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-20T18:01:56.547 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-20T18:01:56.548 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:01:56.548 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-20T18:01:56.549 INFO:tasks.workunit.client.0.trial195.stdout:adding osd6 key to auth repository 2026-02-20T18:01:56.549 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:01:56.862 INFO:tasks.workunit.client.0.trial195.stdout:start osd.6 2026-02-20T18:01:56.863 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-20T18:01:56.863 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:01:56.863 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:01:56.864 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:01:56.864 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:01:56.880 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:56.879+0000 7f2a41b70900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:56.882 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:56.881+0000 7f2a41b70900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:56.883 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:56.882+0000 7f2a41b70900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:01:57.016 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:57.015+0000 7f2a41b70900 -1 Falling back to public interface 2026-02-20T18:01:57.089 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-20T18:01:57.089 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:01:57.089 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-20T18:01:57.089 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:01:57.089 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:01:57.089 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:01:57.090 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:01:57.090 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:01:57.090 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:01:57.090 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-20T18:01:57.313 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:01:57.378 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:01:57.377+0000 7f2a41b70900 -1 osd.6 0 log_to_monitors true 2026-02-20T18:01:58.314 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:01:58.314 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:01:58.315 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:01:58.315 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:01:58.316 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:01:58.316 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-20T18:01:58.540 INFO:tasks.workunit.client.0.trial195.stdout:osd.6 up in weight 1 up_from 52 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/3129149902,v1:127.0.0.1:6851/3129149902] [v2:127.0.0.1:6852/3129149902,v1:127.0.0.1:6853/3129149902] exists,up c316bd4f-9938-4243-aa58-b14c61dedbbc 2026-02-20T18:01:58.542 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:01:58.542 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:01:58.542 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:01:58.542 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-20T18:01:58.542 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-20T18:01:58.542 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-20T18:01:58.543 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:01:58.543 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:01:58.543 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:01:58.543 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.0.asok 2026-02-20T18:01:58.543 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-20T18:01:58.543 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-osd.0.asok log flush 2026-02-20T18:01:58.588 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T18:01:58.592 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-20T18:01:58.595 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T18:01:41.741+0000 7f2aa621b900 0 load: jerasure load: lrc 2026-02-20T18:01:58.596 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:384: TEST_ec_object_attr_read_error: local poolname=pool-jerasure 2026-02-20T18:01:58.596 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:385: TEST_ec_object_attr_read_error: create_erasure_coded_pool pool-jerasure 3 2 2026-02-20T18:01:58.596 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-20T18:01:58.596 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-20T18:01:58.596 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-20T18:01:58.596 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-20T18:01:58.596 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-20T18:01:58.596 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-20T18:01:58.596 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-20T18:01:58.879 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-20T18:01:58.879 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-20T18:01:59.251 INFO:tasks.workunit.client.0.trial195.stderr:pool 'pool-jerasure' already exists 2026-02-20T18:01:59.259 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-20T18:02:00.260 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-20T18:02:00.260 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:02:00.260 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:02:00.260 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:02:00.261 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:02:00.261 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:02:00.261 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:02:00.261 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:02:00.262 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:02:00.262 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:02:00.348 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:02:00.348 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:02:00.348 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:02:00.348 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:02:00.348 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:02:00.348 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:02:00.574 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:02:00.574 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:02:00.574 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:02:00.575 INFO:tasks.workunit.client.0.trial195.stderr:3 2026-02-20T18:02:00.575 INFO:tasks.workunit.client.0.trial195.stderr:4 2026-02-20T18:02:00.575 INFO:tasks.workunit.client.0.trial195.stderr:5 2026-02-20T18:02:00.575 INFO:tasks.workunit.client.0.trial195.stderr:6' 2026-02-20T18:02:00.575 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:02:00.575 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:02:00.575 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:02:00.639 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803782 2026-02-20T18:02:00.639 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803782 2026-02-20T18:02:00.639 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782' 2026-02-20T18:02:00.639 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:02:00.639 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:02:00.704 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607558 2026-02-20T18:02:00.704 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607558 2026-02-20T18:02:00.704 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-51539607558' 2026-02-20T18:02:00.704 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:02:00.704 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:02:00.768 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411333 2026-02-20T18:02:00.768 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411333 2026-02-20T18:02:00.768 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-51539607558 2-77309411333' 2026-02-20T18:02:00.768 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:02:00.768 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:02:00.832 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964116997 2026-02-20T18:02:00.832 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964116997 2026-02-20T18:02:00.832 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-51539607558 2-77309411333 3-115964116997' 2026-02-20T18:02:00.832 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:02:00.832 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-20T18:02:00.896 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855364 2026-02-20T18:02:00.896 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855364 2026-02-20T18:02:00.896 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-51539607558 2-77309411333 3-115964116997 4-150323855364' 2026-02-20T18:02:00.896 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:02:00.896 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-20T18:02:00.962 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=188978561028 2026-02-20T18:02:00.962 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 188978561028 2026-02-20T18:02:00.962 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-51539607558 2-77309411333 3-115964116997 4-150323855364 5-188978561028' 2026-02-20T18:02:00.962 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:02:00.962 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-20T18:02:01.026 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=223338299395 2026-02-20T18:02:01.026 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 223338299395 2026-02-20T18:02:01.026 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-51539607558 2-77309411333 3-115964116997 4-150323855364 5-188978561028 6-223338299395' 2026-02-20T18:02:01.026 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:02:01.027 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803782 2026-02-20T18:02:01.027 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:02:01.027 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:02:01.028 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803782 2026-02-20T18:02:01.028 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:02:01.029 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803782 2026-02-20T18:02:01.029 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803782' 2026-02-20T18:02:01.030 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.0 seq 25769803782 2026-02-20T18:02:01.030 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:02:01.253 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803780 -lt 25769803782 2026-02-20T18:02:01.253 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:02:02.254 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:02:02.255 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:02:02.485 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803782 -lt 25769803782 2026-02-20T18:02:02.485 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:02:02.485 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607558 2026-02-20T18:02:02.485 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:02:02.486 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:02:02.486 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-51539607558 2026-02-20T18:02:02.487 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:02:02.488 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=51539607558 2026-02-20T18:02:02.488 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 51539607558' 2026-02-20T18:02:02.488 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.1 seq 51539607558 2026-02-20T18:02:02.488 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:02:02.719 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 51539607558 -lt 51539607558 2026-02-20T18:02:02.719 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:02:02.719 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411333 2026-02-20T18:02:02.720 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:02:02.720 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:02:02.720 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-77309411333 2026-02-20T18:02:02.721 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:02:02.722 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=77309411333 2026-02-20T18:02:02.722 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 77309411333' 2026-02-20T18:02:02.722 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.2 seq 77309411333 2026-02-20T18:02:02.722 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:02:02.948 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411333 -lt 77309411333 2026-02-20T18:02:02.948 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:02:02.949 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964116997 2026-02-20T18:02:02.949 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:02:02.949 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:02:02.950 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-115964116997 2026-02-20T18:02:02.950 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:02:02.951 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=115964116997 2026-02-20T18:02:02.951 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 115964116997' 2026-02-20T18:02:02.951 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.3 seq 115964116997 2026-02-20T18:02:02.952 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:02:03.174 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 115964116997 -lt 115964116997 2026-02-20T18:02:03.174 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:02:03.175 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-150323855364 2026-02-20T18:02:03.175 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:02:03.175 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-20T18:02:03.176 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-150323855364 2026-02-20T18:02:03.176 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:02:03.177 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=150323855364 2026-02-20T18:02:03.177 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.4 seq 150323855364 2026-02-20T18:02:03.177 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 150323855364' 2026-02-20T18:02:03.178 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-20T18:02:03.407 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 150323855364 -lt 150323855364 2026-02-20T18:02:03.407 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:02:03.408 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-188978561028 2026-02-20T18:02:03.408 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:02:03.408 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-20T18:02:03.409 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-188978561028 2026-02-20T18:02:03.409 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:02:03.410 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=188978561028 2026-02-20T18:02:03.410 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 188978561028' 2026-02-20T18:02:03.411 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.5 seq 188978561028 2026-02-20T18:02:03.411 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-20T18:02:03.634 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 188978561028 -lt 188978561028 2026-02-20T18:02:03.634 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:02:03.636 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-223338299395 2026-02-20T18:02:03.636 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:02:03.636 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-20T18:02:03.636 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-223338299395 2026-02-20T18:02:03.636 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:02:03.637 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=223338299395 2026-02-20T18:02:03.637 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.6 seq 223338299395 2026-02-20T18:02:03.638 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 223338299395' 2026-02-20T18:02:03.638 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-20T18:02:03.860 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 223338299396 -lt 223338299395 2026-02-20T18:02:03.861 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:02:03.861 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:02:03.861 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:02:04.173 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:02:04.173 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:02:04.173 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:02:04.173 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:02:04.174 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:02:04.174 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:02:04.174 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:02:04.174 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:02:04.395 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:02:04.396 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:02:04.396 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:02:04.397 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:02:04.733 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:02:04.733 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:02:04.733 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:02:04.733 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:387: TEST_ec_object_attr_read_error: get_primary pool-jerasure myobject 2026-02-20T18:02:04.734 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=pool-jerasure 2026-02-20T18:02:04.734 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=myobject 2026-02-20T18:02:04.734 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map pool-jerasure myobject 2026-02-20T18:02:04.734 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-20T18:02:04.957 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:387: TEST_ec_object_attr_read_error: local primary_osd=3 2026-02-20T18:02:04.957 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:389: TEST_ec_object_attr_read_error: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-20T18:02:04.957 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:02:04.957 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:02:04.958 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:02:04.958 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:02:04.958 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:02:05.265 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:02:05.265 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:392: TEST_ec_object_attr_read_error: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-20T18:02:05.265 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-20T18:02:05.265 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-20T18:02:05.266 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-20T18:02:05.266 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:02:05.266 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-20T18:02:05.266 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:02:05.266 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-20T18:02:05.266 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:02:05.266 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-20T18:02:05.266 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:02:05.266 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-20T18:02:05.266 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-20T18:02:06.108 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:395: TEST_ec_object_attr_read_error: inject_eio ec mdata pool-jerasure myobject td/test-erasure-eio 1 2026-02-20T18:02:06.109 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=ec 2026-02-20T18:02:06.109 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-20T18:02:06.109 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=mdata 2026-02-20T18:02:06.109 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-20T18:02:06.109 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=pool-jerasure 2026-02-20T18:02:06.109 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-20T18:02:06.109 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=myobject 2026-02-20T18:02:06.109 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-20T18:02:06.109 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/test-erasure-eio 2026-02-20T18:02:06.109 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-20T18:02:06.110 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=1 2026-02-20T18:02:06.110 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-20T18:02:06.110 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds pool-jerasure myobject 2026-02-20T18:02:06.110 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:02:06.110 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-20T18:02:06.110 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-20T18:02:06.110 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:02:06.336 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=2147483647 2026-02-20T18:02:06.336 INFO:tasks.workunit.client.0.trial195.stderr:5 2026-02-20T18:02:06.336 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:02:06.336 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T18:02:06.337 INFO:tasks.workunit.client.0.trial195.stderr:6' 2026-02-20T18:02:06.337 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 2147483647 5 1 0 6 2026-02-20T18:02:06.337 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('2147483647' '5' '1' '0' '6') 2026-02-20T18:02:06.337 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-20T18:02:06.337 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=5 2026-02-20T18:02:06.337 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' ec '!=' ec ']' 2026-02-20T18:02:06.337 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/test-erasure-eio/5/type 2026-02-20T18:02:06.339 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-20T18:02:06.339 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 5 bluestore_debug_inject_read_err true 2026-02-20T18:02:06.339 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-20T18:02:06.339 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=5 2026-02-20T18:02:06.339 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-20T18:02:06.340 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-20T18:02:06.340 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-20T18:02:06.340 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.5 2026-02-20T18:02:06.340 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.5 2026-02-20T18:02:06.340 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.5 ']' 2026-02-20T18:02:06.340 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:02:06.340 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:02:06.340 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:02:06.340 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.5.asok 2026-02-20T18:02:06.340 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.85808/ceph-osd.5.asok config set bluestore_debug_inject_read_err true 2026-02-20T18:02:06.389 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-20T18:02:06.389 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-20T18:02:06.391 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-20T18:02:06.391 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.5 2026-02-20T18:02:06.391 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.5 2026-02-20T18:02:06.391 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.5 ']' 2026-02-20T18:02:06.391 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:02:06.391 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:02:06.391 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:02:06.391 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.5.asok 2026-02-20T18:02:06.391 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-20T18:02:06.392 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-osd.5.asok injectmdataerr pool-jerasure myobject 1 2026-02-20T18:02:06.442 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:398: TEST_ec_object_attr_read_error: activate_osd td/test-erasure-eio 3 2026-02-20T18:02:06.443 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-20T18:02:06.443 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-20T18:02:06.443 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-20T18:02:06.443 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-20T18:02:06.443 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-20T18:02:06.443 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:02:06.443 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:02:06.443 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:02:06.443 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:02:06.443 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-20T18:02:06.444 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-20T18:02:06.444 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-20T18:02:06.444 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-20T18:02:06.444 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:02:06.444 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-20T18:02:06.444 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:02:06.444 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:02:06.444 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:02:06.444 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:02:06.444 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:02:06.444 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:02:06.444 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:02:06.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:02:06.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:02:06.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:02:06.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:02:06.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:02:06.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:02:06.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:02:06.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-20T18:02:06.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-20T18:02:06.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-20T18:02:06.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-20T18:02:06.446 INFO:tasks.workunit.client.0.trial195.stdout:start osd.3 2026-02-20T18:02:06.447 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:02:06.447 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-20T18:02:06.447 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-20T18:02:06.447 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-20T18:02:06.447 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-20T18:02:06.447 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-20T18:02:06.458 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:06.457+0000 7fa28f739900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:06.459 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:06.459+0000 7fa28f739900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:06.460 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:06.460+0000 7fa28f739900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:06.593 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:06.591+0000 7fa28f739900 -1 Falling back to public interface 2026-02-20T18:02:06.666 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-20T18:02:06.666 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:02:06.666 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-20T18:02:06.666 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:02:06.666 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:02:06.666 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:02:06.667 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:02:06.667 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:02:06.667 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:02:06.667 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T18:02:06.898 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:02:06.975 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:06.974+0000 7fa28f739900 -1 osd.3 59 log_to_monitors true 2026-02-20T18:02:07.899 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:02:07.900 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:02:07.900 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:02:07.901 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:02:07.901 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:02:07.901 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T18:02:08.128 INFO:tasks.workunit.client.0.trial195.stdout:osd.3 up in weight 1 up_from 63 up_thru 63 down_at 60 last_clean_interval [27,59) [v2:127.0.0.1:6826/1657431985,v1:127.0.0.1:6827/1657431985] [v2:127.0.0.1:6828/1657431985,v1:127.0.0.1:6829/1657431985] exists,up 3dd0db4b-4be8-4deb-8812-b25061fc9d5b 2026-02-20T18:02:08.130 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:02:08.130 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:02:08.130 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:02:08.130 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:401: TEST_ec_object_attr_read_error: wait_for_clean 2026-02-20T18:02:08.130 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:02:08.131 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:02:08.131 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:02:08.131 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:02:08.131 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:02:08.131 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:02:08.131 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:02:08.131 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:02:08.131 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:02:08.215 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:02:08.215 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:02:08.215 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:02:08.216 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:02:08.216 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:02:08.216 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:02:08.440 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:02:08.441 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:02:08.441 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:02:08.441 INFO:tasks.workunit.client.0.trial195.stderr:3 2026-02-20T18:02:08.441 INFO:tasks.workunit.client.0.trial195.stderr:4 2026-02-20T18:02:08.441 INFO:tasks.workunit.client.0.trial195.stderr:5 2026-02-20T18:02:08.441 INFO:tasks.workunit.client.0.trial195.stderr:6' 2026-02-20T18:02:08.441 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:02:08.441 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:02:08.441 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:02:08.506 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803786 2026-02-20T18:02:08.506 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803786 2026-02-20T18:02:08.506 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786' 2026-02-20T18:02:08.507 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:02:08.507 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:02:08.573 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607561 2026-02-20T18:02:08.573 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607561 2026-02-20T18:02:08.574 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786 1-51539607561' 2026-02-20T18:02:08.574 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:02:08.574 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:02:08.639 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411337 2026-02-20T18:02:08.639 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411337 2026-02-20T18:02:08.640 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786 1-51539607561 2-77309411337' 2026-02-20T18:02:08.640 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:02:08.640 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:02:08.705 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=270582939651 2026-02-20T18:02:08.705 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 270582939651 2026-02-20T18:02:08.705 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786 1-51539607561 2-77309411337 3-270582939651' 2026-02-20T18:02:08.705 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:02:08.705 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-20T18:02:08.767 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855368 2026-02-20T18:02:08.767 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855368 2026-02-20T18:02:08.767 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786 1-51539607561 2-77309411337 3-270582939651 4-150323855368' 2026-02-20T18:02:08.767 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:02:08.767 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-20T18:02:08.831 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=188978561031 2026-02-20T18:02:08.831 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 188978561031 2026-02-20T18:02:08.832 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786 1-51539607561 2-77309411337 3-270582939651 4-150323855368 5-188978561031' 2026-02-20T18:02:08.832 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:02:08.832 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-20T18:02:08.896 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=223338299399 2026-02-20T18:02:08.896 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 223338299399 2026-02-20T18:02:08.896 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786 1-51539607561 2-77309411337 3-270582939651 4-150323855368 5-188978561031 6-223338299399' 2026-02-20T18:02:08.896 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:02:08.896 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803786 2026-02-20T18:02:08.896 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:02:08.897 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:02:08.898 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803786 2026-02-20T18:02:08.898 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:02:08.899 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803786 2026-02-20T18:02:08.899 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803786' 2026-02-20T18:02:08.899 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.0 seq 25769803786 2026-02-20T18:02:08.900 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:02:09.128 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803783 -lt 25769803786 2026-02-20T18:02:09.128 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:02:10.129 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:02:10.130 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:02:10.353 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803786 -lt 25769803786 2026-02-20T18:02:10.353 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:02:10.354 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607561 2026-02-20T18:02:10.354 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:02:10.354 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:02:10.355 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-51539607561 2026-02-20T18:02:10.355 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:02:10.356 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=51539607561 2026-02-20T18:02:10.356 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 51539607561' 2026-02-20T18:02:10.357 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.1 seq 51539607561 2026-02-20T18:02:10.357 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:02:10.582 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 51539607561 -lt 51539607561 2026-02-20T18:02:10.582 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:02:10.582 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411337 2026-02-20T18:02:10.582 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:02:10.583 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:02:10.583 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-77309411337 2026-02-20T18:02:10.584 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:02:10.584 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=77309411337 2026-02-20T18:02:10.584 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 77309411337' 2026-02-20T18:02:10.585 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.2 seq 77309411337 2026-02-20T18:02:10.585 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:02:10.811 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411337 -lt 77309411337 2026-02-20T18:02:10.811 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:02:10.812 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-270582939651 2026-02-20T18:02:10.812 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:02:10.812 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:02:10.813 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-270582939651 2026-02-20T18:02:10.813 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:02:10.814 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=270582939651 2026-02-20T18:02:10.814 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 270582939651' 2026-02-20T18:02:10.815 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.3 seq 270582939651 2026-02-20T18:02:10.815 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:02:11.041 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 270582939651 -lt 270582939651 2026-02-20T18:02:11.041 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:02:11.042 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-150323855368 2026-02-20T18:02:11.042 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:02:11.043 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-20T18:02:11.043 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-150323855368 2026-02-20T18:02:11.043 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:02:11.044 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=150323855368 2026-02-20T18:02:11.044 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 150323855368' 2026-02-20T18:02:11.045 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.4 seq 150323855368 2026-02-20T18:02:11.045 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-20T18:02:11.271 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 150323855368 -lt 150323855368 2026-02-20T18:02:11.271 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:02:11.272 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-188978561031 2026-02-20T18:02:11.272 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:02:11.272 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-20T18:02:11.273 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-188978561031 2026-02-20T18:02:11.273 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:02:11.274 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=188978561031 2026-02-20T18:02:11.274 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 188978561031' 2026-02-20T18:02:11.274 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.5 seq 188978561031 2026-02-20T18:02:11.275 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-20T18:02:11.500 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 188978561031 -lt 188978561031 2026-02-20T18:02:11.500 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:02:11.501 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-223338299399 2026-02-20T18:02:11.501 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:02:11.501 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-20T18:02:11.502 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-223338299399 2026-02-20T18:02:11.502 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:02:11.503 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=223338299399 2026-02-20T18:02:11.504 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 223338299399' 2026-02-20T18:02:11.504 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.6 seq 223338299399 2026-02-20T18:02:11.505 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-20T18:02:11.730 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 223338299399 -lt 223338299399 2026-02-20T18:02:11.731 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:02:11.731 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:02:11.731 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:02:12.041 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:02:12.042 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:02:12.042 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:02:12.042 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:02:12.042 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:02:12.042 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:02:12.042 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:02:12.042 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:02:12.266 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:02:12.266 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:02:12.267 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:02:12.267 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:02:12.585 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:02:12.585 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:02:12.585 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:02:12.585 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:403: TEST_ec_object_attr_read_error: rados_get td/test-erasure-eio pool-jerasure myobject 2026-02-20T18:02:12.585 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-20T18:02:12.585 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-20T18:02:12.585 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=myobject 2026-02-20T18:02:12.586 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-20T18:02:12.586 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-20T18:02:12.586 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get myobject td/test-erasure-eio/COPY 2026-02-20T18:02:12.604 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-20T18:02:12.605 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-20T18:02:12.606 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:405: TEST_ec_object_attr_read_error: delete_erasure_coded_pool pool-jerasure 2026-02-20T18:02:12.606 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-20T18:02:12.606 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-20T18:02:12.882 INFO:tasks.workunit.client.0.trial195.stderr:pool 'pool-jerasure' does not exist 2026-02-20T18:02:12.890 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-20T18:02:13.167 INFO:tasks.workunit.client.0.trial195.stderr:erasure-code-profile myprofile does not exist 2026-02-20T18:02:13.177 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-20T18:02:13.177 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-20T18:02:13.177 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T18:02:13.177 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-20T18:02:13.177 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:02:13.177 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:02:13.178 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:02:13.178 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:02:13.178 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:02:13.287 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:02:13.287 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T18:02:13.288 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T18:02:13.288 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T18:02:13.289 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T18:02:13.289 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T18:02:13.290 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T18:02:13.290 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:02:13.291 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T18:02:13.291 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T18:02:13.291 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:02:13.292 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T18:02:13.293 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T18:02:13.293 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-20T18:02:13.307 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T18:02:13.307 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:02:13.307 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:02:13.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.85808 2026-02-20T18:02:13.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T18:02:13.310 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T18:02:13.310 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-20T18:02:13.310 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-20T18:02:13.310 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-20T18:02:13.310 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-20T18:02:13.310 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-20T18:02:13.310 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T18:02:13.310 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-20T18:02:13.310 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:02:13.310 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:02:13.311 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:02:13.311 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:02:13.311 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:02:13.311 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:02:13.311 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T18:02:13.312 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T18:02:13.312 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T18:02:13.313 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T18:02:13.313 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T18:02:13.313 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T18:02:13.314 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:02:13.314 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T18:02:13.314 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T18:02:13.315 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:02:13.315 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T18:02:13.316 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T18:02:13.316 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-20T18:02:13.317 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T18:02:13.317 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:02:13.318 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:02:13.318 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.85808 2026-02-20T18:02:13.318 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T18:02:13.318 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T18:02:13.318 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-20T18:02:13.319 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-20T18:02:13.320 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:02:13.320 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:02:13.320 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.85808 2026-02-20T18:02:13.321 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-20T18:02:13.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-20T18:02:13.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-20T18:02:13.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-20T18:02:13.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-20T18:02:13.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-20T18:02:13.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-20T18:02:13.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-20T18:02:13.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-20T18:02:13.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-20T18:02:13.323 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-20T18:02:13.342 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-20T18:02:13.343 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:02:13.343 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:02:13.343 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:02:13.343 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:02:13.343 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:02:13.343 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:02:13.343 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-20T18:02:13.369 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-20T18:02:13.371 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-20T18:02:13.371 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T18:02:13.371 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T18:02:13.371 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-20T18:02:13.371 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-20T18:02:13.371 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T18:02:13.371 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:02:13.371 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:02:13.371 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:02:13.371 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:02:13.371 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:02:13.372 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:02:13.372 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T18:02:13.372 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.85808/ceph-mon.a.asok config get fsid 2026-02-20T18:02:13.416 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-20T18:02:13.416 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T18:02:13.416 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T18:02:13.416 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-20T18:02:13.417 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-20T18:02:13.418 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T18:02:13.418 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:02:13.418 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:02:13.418 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:02:13.418 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:02:13.418 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:02:13.418 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:02:13.418 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T18:02:13.419 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.85808/ceph-mon.a.asok config get mon_host 2026-02-20T18:02:13.461 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-20T18:02:13.461 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-20T18:02:13.461 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-20T18:02:13.462 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-20T18:02:13.462 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-20T18:02:13.462 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-20T18:02:13.462 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-20T18:02:13.560 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-20T18:02:13.560 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:02:13.560 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:02:13.560 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:02:13.560 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:02:13.560 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:02:13.560 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:02:13.561 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T18:02:13.561 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T18:02:13.579 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-20T18:02:13.579 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-20T18:02:13.676 INFO:tasks.workunit.client.0.trial195.stderr:pool 'rbd' already exists 2026-02-20T18:02:13.682 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-20T18:02:14.684 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-20T18:02:14.684 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:02:14.684 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:02:14.684 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:02:14.684 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:02:14.684 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:02:14.684 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:02:14.684 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-20T18:02:14.684 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-mon.a.asok log flush 2026-02-20T18:02:14.727 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T18:02:14.731 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-20T18:02:14.733 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T18:02:13.361+0000 7f95ed412d80 0 load: jerasure load: lrc 2026-02-20T18:02:14.733 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_recovery_multiple_errors td/test-erasure-eio 2026-02-20T18:02:14.734 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:438: TEST_ec_recovery_multiple_errors: local dir=td/test-erasure-eio 2026-02-20T18:02:14.734 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:439: TEST_ec_recovery_multiple_errors: local objname=myobject 2026-02-20T18:02:14.734 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:441: TEST_ec_recovery_multiple_errors: setup_osds 9 2026-02-20T18:02:14.734 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=9 2026-02-20T18:02:14.734 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-20T18:02:14.734 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 9 - 1 2026-02-20T18:02:14.734 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 8 2026-02-20T18:02:14.737 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:02:14.737 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-20T18:02:14.737 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:02:14.737 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:02:14.737 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-20T18:02:14.737 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:02:14.737 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-20T18:02:14.737 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:02:14.737 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:02:14.738 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:02:14.738 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:02:14.738 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-20T18:02:14.738 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-20T18:02:14.738 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:02:14.738 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:02:14.738 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:02:14.738 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:02:14.738 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:02:14.738 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:02:14.738 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:02:14.738 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:02:14.738 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:02:14.738 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:02:14.738 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:02:14.738 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:02:14.738 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:02:14.738 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:02:14.739 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:02:14.739 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:02:14.739 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:02:14.739 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:02:14.739 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:02:14.739 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:02:14.739 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:02:14.739 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:02:14.739 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-20T18:02:14.739 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:02:14.740 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b6a65ea3-f33e-445e-8fe7-41e02e1a180e 2026-02-20T18:02:14.740 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 b6a65ea3-f33e-445e-8fe7-41e02e1a180e' 2026-02-20T18:02:14.740 INFO:tasks.workunit.client.0.trial195.stdout:add osd0 b6a65ea3-f33e-445e-8fe7-41e02e1a180e 2026-02-20T18:02:14.740 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:02:14.750 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCmoZhpgOamLBAAF7TY03rb4Cmz1j9WODIV7Q== 2026-02-20T18:02:14.750 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCmoZhpgOamLBAAF7TY03rb4Cmz1j9WODIV7Q=="}' 2026-02-20T18:02:14.750 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b6a65ea3-f33e-445e-8fe7-41e02e1a180e -i td/test-erasure-eio/0/new.json 2026-02-20T18:02:14.854 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:02:14.860 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-20T18:02:14.861 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCmoZhpgOamLBAAF7TY03rb4Cmz1j9WODIV7Q== --osd-uuid b6a65ea3-f33e-445e-8fe7-41e02e1a180e 2026-02-20T18:02:14.879 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:14.879+0000 7f0fa97f0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:14.881 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:14.880+0000 7f0fa97f0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:14.882 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:14.881+0000 7f0fa97f0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:14.882 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:14.881+0000 7f0fa97f0900 -1 bdev(0x5571c2594c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-20T18:02:14.883 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:14.882+0000 7f0fa97f0900 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-20T18:02:15.311 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-20T18:02:15.311 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:02:15.312 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-20T18:02:15.312 INFO:tasks.workunit.client.0.trial195.stdout:adding osd0 key to auth repository 2026-02-20T18:02:15.312 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:02:15.416 INFO:tasks.workunit.client.0.trial195.stdout:start osd.0 2026-02-20T18:02:15.417 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-20T18:02:15.417 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:02:15.417 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:02:15.418 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:02:15.418 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:02:15.434 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:15.432+0000 7f1ce1497900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:15.436 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:15.434+0000 7f1ce1497900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:15.437 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:15.435+0000 7f1ce1497900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:15.520 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-20T18:02:15.520 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:02:15.520 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-20T18:02:15.520 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:02:15.521 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:02:15.521 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:02:15.521 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:02:15.521 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:02:15.522 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:02:15.522 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T18:02:15.621 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:02:15.663 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:15.662+0000 7f1ce1497900 -1 Falling back to public interface 2026-02-20T18:02:15.916 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:15.914+0000 7f1ce1497900 -1 osd.0 0 log_to_monitors true 2026-02-20T18:02:16.622 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:02:16.622 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:02:16.623 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:02:16.623 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:02:16.623 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:02:16.623 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T18:02:16.848 INFO:tasks.workunit.client.0.trial195.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2255490607,v1:127.0.0.1:6803/2255490607] [v2:127.0.0.1:6804/2255490607,v1:127.0.0.1:6805/2255490607] exists,up b6a65ea3-f33e-445e-8fe7-41e02e1a180e 2026-02-20T18:02:16.850 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:02:16.850 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:02:16.850 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:02:16.850 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:02:16.850 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-20T18:02:16.850 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:02:16.850 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:02:16.850 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-20T18:02:16.850 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:02:16.850 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-20T18:02:16.850 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:02:16.850 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:02:16.850 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:02:16.850 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:02:16.850 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-20T18:02:16.850 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-20T18:02:16.850 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:02:16.850 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:02:16.851 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:02:16.851 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:02:16.851 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:02:16.851 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:02:16.851 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:02:16.851 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:02:16.851 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:02:16.852 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:02:16.852 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:02:16.852 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:02:16.852 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:02:16.852 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:02:16.852 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:02:16.852 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:02:16.852 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:02:16.852 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:02:16.852 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:02:16.852 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:02:16.852 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:02:16.852 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:02:16.852 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-20T18:02:16.852 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:02:16.853 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=dc133da2-68c3-48a9-9d7d-01b9fa317835 2026-02-20T18:02:16.853 INFO:tasks.workunit.client.0.trial195.stdout:add osd1 dc133da2-68c3-48a9-9d7d-01b9fa317835 2026-02-20T18:02:16.853 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 dc133da2-68c3-48a9-9d7d-01b9fa317835' 2026-02-20T18:02:16.853 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:02:16.865 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCooZhpJVKGMxAAhXDKJcK8RVWG3J4Jz6L7Dw== 2026-02-20T18:02:16.865 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCooZhpJVKGMxAAhXDKJcK8RVWG3J4Jz6L7Dw=="}' 2026-02-20T18:02:16.865 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new dc133da2-68c3-48a9-9d7d-01b9fa317835 -i td/test-erasure-eio/1/new.json 2026-02-20T18:02:17.093 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:02:17.101 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-20T18:02:17.103 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCooZhpJVKGMxAAhXDKJcK8RVWG3J4Jz6L7Dw== --osd-uuid dc133da2-68c3-48a9-9d7d-01b9fa317835 2026-02-20T18:02:17.121 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:17.119+0000 7f5135db0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:17.122 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:17.120+0000 7f5135db0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:17.123 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:17.121+0000 7f5135db0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:17.124 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:17.122+0000 7f5135db0900 -1 bdev(0x55854fb56c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-20T18:02:17.124 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:17.122+0000 7f5135db0900 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-20T18:02:17.444 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-20T18:02:17.444 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:02:17.444 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-20T18:02:17.445 INFO:tasks.workunit.client.0.trial195.stdout:adding osd1 key to auth repository 2026-02-20T18:02:17.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:02:17.760 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-20T18:02:17.760 INFO:tasks.workunit.client.0.trial195.stdout:start osd.1 2026-02-20T18:02:17.761 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:02:17.761 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:02:17.761 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:02:17.761 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:02:17.777 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:17.775+0000 7f02921f2900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:17.778 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:17.777+0000 7f02921f2900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:17.779 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:17.778+0000 7f02921f2900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:17.892 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:17.890+0000 7f02921f2900 -1 Falling back to public interface 2026-02-20T18:02:17.984 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-20T18:02:17.984 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:02:17.984 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-20T18:02:17.984 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:02:17.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:02:17.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:02:17.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:02:17.985 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:02:17.986 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:02:17.986 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:02:18.205 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:18.203+0000 7f02921f2900 -1 osd.1 0 log_to_monitors true 2026-02-20T18:02:18.210 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:02:19.211 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:02:19.212 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:02:19.212 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:02:19.213 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:02:19.213 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:02:19.213 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:02:19.437 INFO:tasks.workunit.client.0.trial195.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/4129741491,v1:127.0.0.1:6811/4129741491] [v2:127.0.0.1:6812/4129741491,v1:127.0.0.1:6813/4129741491] exists,up dc133da2-68c3-48a9-9d7d-01b9fa317835 2026-02-20T18:02:19.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:02:19.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:02:19.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:02:19.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:02:19.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-20T18:02:19.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:02:19.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:02:19.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-20T18:02:19.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:02:19.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-20T18:02:19.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:02:19.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:02:19.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:02:19.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:02:19.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-20T18:02:19.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-20T18:02:19.439 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:02:19.439 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:02:19.439 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:02:19.439 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:02:19.439 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:02:19.439 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:02:19.439 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:02:19.439 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:02:19.439 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:02:19.439 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:02:19.439 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:02:19.439 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:02:19.439 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:02:19.439 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:02:19.439 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:02:19.440 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:02:19.440 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:02:19.440 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:02:19.440 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:02:19.440 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:02:19.440 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:02:19.440 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:02:19.440 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-20T18:02:19.440 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:02:19.441 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2809f39f-4091-4819-9fc2-549e3b3758c5 2026-02-20T18:02:19.441 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 2809f39f-4091-4819-9fc2-549e3b3758c5' 2026-02-20T18:02:19.441 INFO:tasks.workunit.client.0.trial195.stdout:add osd2 2809f39f-4091-4819-9fc2-549e3b3758c5 2026-02-20T18:02:19.441 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:02:19.453 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCroZhpiyn7GhAAwMxUgOkQpiJA9tLhy2eZJQ== 2026-02-20T18:02:19.453 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCroZhpiyn7GhAAwMxUgOkQpiJA9tLhy2eZJQ=="}' 2026-02-20T18:02:19.453 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2809f39f-4091-4819-9fc2-549e3b3758c5 -i td/test-erasure-eio/2/new.json 2026-02-20T18:02:19.682 INFO:tasks.workunit.client.0.trial195.stdout:2 2026-02-20T18:02:19.690 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-20T18:02:19.691 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCroZhpiyn7GhAAwMxUgOkQpiJA9tLhy2eZJQ== --osd-uuid 2809f39f-4091-4819-9fc2-549e3b3758c5 2026-02-20T18:02:19.709 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:19.707+0000 7ffbc98a8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:19.711 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:19.709+0000 7ffbc98a8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:19.712 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:19.710+0000 7ffbc98a8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:19.712 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:19.710+0000 7ffbc98a8900 -1 bdev(0x55df7f706c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-20T18:02:19.713 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:19.710+0000 7ffbc98a8900 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-20T18:02:20.056 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-20T18:02:20.056 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:02:20.057 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-20T18:02:20.057 INFO:tasks.workunit.client.0.trial195.stdout:adding osd2 key to auth repository 2026-02-20T18:02:20.057 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:02:20.379 INFO:tasks.workunit.client.0.trial195.stdout:start osd.2 2026-02-20T18:02:20.380 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-20T18:02:20.380 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:02:20.380 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:02:20.380 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:02:20.380 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:02:20.396 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:20.394+0000 7f955d881900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:20.398 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:20.396+0000 7f955d881900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:20.399 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:20.397+0000 7f955d881900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:20.556 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:20.554+0000 7f955d881900 -1 Falling back to public interface 2026-02-20T18:02:20.605 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-20T18:02:20.605 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:02:20.605 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-20T18:02:20.605 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:02:20.605 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:02:20.605 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:02:20.605 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:02:20.606 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:02:20.606 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:02:20.606 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:02:20.829 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:02:20.898 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:20.896+0000 7f955d881900 -1 osd.2 0 log_to_monitors true 2026-02-20T18:02:21.830 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:02:21.830 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:02:21.830 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:02:21.830 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:02:21.830 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:02:21.830 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:02:22.052 INFO:tasks.workunit.client.0.trial195.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1343502802,v1:127.0.0.1:6819/1343502802] [v2:127.0.0.1:6820/1343502802,v1:127.0.0.1:6821/1343502802] exists,up 2809f39f-4091-4819-9fc2-549e3b3758c5 2026-02-20T18:02:22.056 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:02:22.057 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:02:22.057 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:02:22.057 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:02:22.057 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-20T18:02:22.057 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:02:22.057 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:02:22.057 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-20T18:02:22.057 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:02:22.057 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-20T18:02:22.057 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:02:22.057 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:02:22.058 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:02:22.058 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:02:22.058 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-20T18:02:22.058 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-20T18:02:22.058 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:02:22.058 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:02:22.058 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:02:22.058 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:02:22.058 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:02:22.058 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:02:22.059 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:02:22.059 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:02:22.059 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:02:22.059 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:02:22.059 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:02:22.059 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:02:22.059 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:02:22.059 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:02:22.059 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:02:22.059 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:02:22.059 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:02:22.059 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:02:22.060 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:02:22.060 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:02:22.060 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:02:22.060 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:02:22.060 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-20T18:02:22.060 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:02:22.060 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=13e4f62f-07f3-4ea8-9b31-100527fe359b 2026-02-20T18:02:22.060 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 13e4f62f-07f3-4ea8-9b31-100527fe359b' 2026-02-20T18:02:22.061 INFO:tasks.workunit.client.0.trial195.stdout:add osd3 13e4f62f-07f3-4ea8-9b31-100527fe359b 2026-02-20T18:02:22.061 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:02:22.068 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCuoZhpl5MIBBAAIVb79OWFdb1XJCBVJFD/jA== 2026-02-20T18:02:22.069 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCuoZhpl5MIBBAAIVb79OWFdb1XJCBVJFD/jA=="}' 2026-02-20T18:02:22.069 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 13e4f62f-07f3-4ea8-9b31-100527fe359b -i td/test-erasure-eio/3/new.json 2026-02-20T18:02:22.298 INFO:tasks.workunit.client.0.trial195.stdout:3 2026-02-20T18:02:22.306 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-20T18:02:22.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCuoZhpl5MIBBAAIVb79OWFdb1XJCBVJFD/jA== --osd-uuid 13e4f62f-07f3-4ea8-9b31-100527fe359b 2026-02-20T18:02:22.325 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:22.323+0000 7fc2f0781900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:22.326 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:22.324+0000 7fc2f0781900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:22.327 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:22.325+0000 7fc2f0781900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:22.328 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:22.326+0000 7fc2f0781900 -1 bdev(0x55b72b6fec00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-20T18:02:22.328 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:22.326+0000 7fc2f0781900 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-20T18:02:22.647 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-20T18:02:22.648 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:02:22.648 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-20T18:02:22.649 INFO:tasks.workunit.client.0.trial195.stdout:adding osd3 key to auth repository 2026-02-20T18:02:22.649 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:02:22.963 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-20T18:02:22.964 INFO:tasks.workunit.client.0.trial195.stdout:start osd.3 2026-02-20T18:02:22.964 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:02:22.965 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:02:22.965 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:02:22.965 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:02:22.981 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:22.979+0000 7fe76f5b0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:22.982 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:22.980+0000 7fe76f5b0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:22.983 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:22.981+0000 7fe76f5b0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:23.111 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:23.109+0000 7fe76f5b0900 -1 Falling back to public interface 2026-02-20T18:02:23.187 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-20T18:02:23.187 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:02:23.187 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-20T18:02:23.188 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:02:23.188 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:02:23.188 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:02:23.188 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:02:23.188 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:02:23.189 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:02:23.189 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T18:02:23.413 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:02:23.417 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:23.415+0000 7fe76f5b0900 -1 osd.3 0 log_to_monitors true 2026-02-20T18:02:24.414 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:02:24.414 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:02:24.415 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:02:24.415 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:02:24.415 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:02:24.415 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T18:02:24.639 INFO:tasks.workunit.client.0.trial195.stdout:osd.3 up in weight 1 up_from 27 up_thru 28 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/4044870798,v1:127.0.0.1:6827/4044870798] [v2:127.0.0.1:6828/4044870798,v1:127.0.0.1:6829/4044870798] exists,up 13e4f62f-07f3-4ea8-9b31-100527fe359b 2026-02-20T18:02:24.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:02:24.644 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:02:24.644 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:02:24.644 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:02:24.644 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-20T18:02:24.644 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:02:24.644 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:02:24.644 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-20T18:02:24.644 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:02:24.644 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-20T18:02:24.644 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:02:24.644 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:02:24.645 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:02:24.645 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:02:24.645 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-20T18:02:24.645 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-20T18:02:24.645 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:02:24.645 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:02:24.645 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:02:24.645 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:02:24.645 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:02:24.645 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:02:24.645 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:02:24.645 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:02:24.645 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:02:24.646 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:02:24.646 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:02:24.646 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:02:24.646 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:02:24.646 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:02:24.646 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:02:24.646 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:02:24.646 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:02:24.646 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:02:24.646 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:02:24.646 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:02:24.646 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:02:24.647 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:02:24.647 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-20T18:02:24.647 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:02:24.647 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=424b1df0-43c6-46f8-91b3-311c9338c9db 2026-02-20T18:02:24.647 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 424b1df0-43c6-46f8-91b3-311c9338c9db' 2026-02-20T18:02:24.647 INFO:tasks.workunit.client.0.trial195.stdout:add osd4 424b1df0-43c6-46f8-91b3-311c9338c9db 2026-02-20T18:02:24.648 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:02:24.655 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCwoZhpdRoAJxAAJTnsjrbr9DKitdx6cmKW7w== 2026-02-20T18:02:24.655 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCwoZhpdRoAJxAAJTnsjrbr9DKitdx6cmKW7w=="}' 2026-02-20T18:02:24.655 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 424b1df0-43c6-46f8-91b3-311c9338c9db -i td/test-erasure-eio/4/new.json 2026-02-20T18:02:24.884 INFO:tasks.workunit.client.0.trial195.stdout:4 2026-02-20T18:02:24.891 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-20T18:02:24.892 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCwoZhpdRoAJxAAJTnsjrbr9DKitdx6cmKW7w== --osd-uuid 424b1df0-43c6-46f8-91b3-311c9338c9db 2026-02-20T18:02:24.910 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:24.908+0000 7fee4c610900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:24.911 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:24.909+0000 7fee4c610900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:24.912 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:24.910+0000 7fee4c610900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:24.912 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:24.911+0000 7fee4c610900 -1 bdev(0x558b1f600c00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-20T18:02:24.913 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:24.911+0000 7fee4c610900 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-20T18:02:25.233 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-20T18:02:25.233 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:02:25.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-20T18:02:25.234 INFO:tasks.workunit.client.0.trial195.stdout:adding osd4 key to auth repository 2026-02-20T18:02:25.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:02:25.553 INFO:tasks.workunit.client.0.trial195.stdout:start osd.4 2026-02-20T18:02:25.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-20T18:02:25.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:02:25.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:02:25.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:02:25.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:02:25.570 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:25.568+0000 7f227efe0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:25.571 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:25.570+0000 7f227efe0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:25.572 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:25.571+0000 7f227efe0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:25.699 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:25.697+0000 7f227efe0900 -1 Falling back to public interface 2026-02-20T18:02:25.781 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-20T18:02:25.782 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:02:25.782 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-20T18:02:25.782 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:02:25.782 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:02:25.782 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:02:25.782 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:02:25.782 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:02:25.783 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:02:25.783 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-20T18:02:26.008 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:02:26.012 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:26.010+0000 7f227efe0900 -1 osd.4 0 log_to_monitors true 2026-02-20T18:02:27.009 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:02:27.009 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:02:27.009 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:02:27.010 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:02:27.010 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:02:27.011 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-20T18:02:27.234 INFO:tasks.workunit.client.0.trial195.stdout:osd.4 up in weight 1 up_from 36 up_thru 37 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/2093160417,v1:127.0.0.1:6835/2093160417] [v2:127.0.0.1:6836/2093160417,v1:127.0.0.1:6837/2093160417] exists,up 424b1df0-43c6-46f8-91b3-311c9338c9db 2026-02-20T18:02:27.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:02:27.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:02:27.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:02:27.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:02:27.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-20T18:02:27.238 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:02:27.238 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:02:27.238 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-20T18:02:27.238 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:02:27.238 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-20T18:02:27.238 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:02:27.238 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:02:27.238 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:02:27.238 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:02:27.238 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-20T18:02:27.238 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-20T18:02:27.238 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:02:27.239 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:02:27.239 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:02:27.239 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:02:27.239 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:02:27.239 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:02:27.239 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:02:27.239 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:02:27.239 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:02:27.239 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:02:27.239 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:02:27.239 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:02:27.239 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:02:27.240 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:02:27.240 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:02:27.240 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:02:27.240 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:02:27.240 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:02:27.240 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:02:27.240 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:02:27.240 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:02:27.240 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:02:27.240 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-20T18:02:27.240 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:02:27.241 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=48fdd8fe-dd18-474d-8244-739e7d01d799 2026-02-20T18:02:27.241 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 48fdd8fe-dd18-474d-8244-739e7d01d799' 2026-02-20T18:02:27.241 INFO:tasks.workunit.client.0.trial195.stdout:add osd5 48fdd8fe-dd18-474d-8244-739e7d01d799 2026-02-20T18:02:27.242 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:02:27.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCzoZhp8kfUDhAAMfLJk4KQsisLDotQgepvWg== 2026-02-20T18:02:27.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCzoZhp8kfUDhAAMfLJk4KQsisLDotQgepvWg=="}' 2026-02-20T18:02:27.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 48fdd8fe-dd18-474d-8244-739e7d01d799 -i td/test-erasure-eio/5/new.json 2026-02-20T18:02:27.477 INFO:tasks.workunit.client.0.trial195.stdout:5 2026-02-20T18:02:27.485 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-20T18:02:27.486 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCzoZhp8kfUDhAAMfLJk4KQsisLDotQgepvWg== --osd-uuid 48fdd8fe-dd18-474d-8244-739e7d01d799 2026-02-20T18:02:27.504 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:27.502+0000 7f6734fe0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:27.505 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:27.503+0000 7f6734fe0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:27.507 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:27.504+0000 7f6734fe0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:27.507 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:27.505+0000 7f6734fe0900 -1 bdev(0x557e27768c00 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-20T18:02:27.507 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:27.505+0000 7f6734fe0900 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-20T18:02:27.825 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-20T18:02:27.826 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:02:27.827 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-20T18:02:27.827 INFO:tasks.workunit.client.0.trial195.stdout:adding osd5 key to auth repository 2026-02-20T18:02:27.828 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:02:28.142 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-20T18:02:28.143 INFO:tasks.workunit.client.0.trial195.stdout:start osd.5 2026-02-20T18:02:28.144 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:02:28.144 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:02:28.144 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:02:28.144 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:02:28.159 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:28.157+0000 7ff059bde900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:28.160 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:28.159+0000 7ff059bde900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:28.161 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:28.160+0000 7ff059bde900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:28.304 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:28.302+0000 7ff059bde900 -1 Falling back to public interface 2026-02-20T18:02:28.368 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-20T18:02:28.368 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:02:28.368 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-20T18:02:28.368 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:02:28.368 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:02:28.369 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:02:28.369 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:02:28.369 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:02:28.370 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:02:28.370 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-20T18:02:28.593 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:02:28.665 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:28.663+0000 7ff059bde900 -1 osd.5 0 log_to_monitors true 2026-02-20T18:02:29.594 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:02:29.594 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:02:29.594 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:02:29.595 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:02:29.595 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:02:29.595 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-20T18:02:29.821 INFO:tasks.workunit.client.0.trial195.stdout:osd.5 up in weight 1 up_from 44 up_thru 45 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/1567206312,v1:127.0.0.1:6843/1567206312] [v2:127.0.0.1:6844/1567206312,v1:127.0.0.1:6845/1567206312] exists,up 48fdd8fe-dd18-474d-8244-739e7d01d799 2026-02-20T18:02:29.825 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:02:29.825 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:02:29.826 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:02:29.826 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:02:29.826 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-20T18:02:29.826 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:02:29.826 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:02:29.826 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-20T18:02:29.826 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:02:29.826 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-20T18:02:29.826 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:02:29.826 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:02:29.826 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:02:29.827 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:02:29.827 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-20T18:02:29.827 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-20T18:02:29.827 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:02:29.827 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:02:29.827 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:02:29.827 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:02:29.827 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:02:29.827 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:02:29.827 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:02:29.827 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:02:29.827 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:02:29.828 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:02:29.828 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:02:29.828 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:02:29.828 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:02:29.828 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:02:29.828 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:02:29.828 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:02:29.828 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:02:29.828 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:02:29.828 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:02:29.828 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:02:29.828 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:02:29.829 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:02:29.829 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-20T18:02:29.829 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:02:29.829 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=7627652f-5ff6-4df6-868c-ab3170a7c5d8 2026-02-20T18:02:29.829 INFO:tasks.workunit.client.0.trial195.stdout:add osd6 7627652f-5ff6-4df6-868c-ab3170a7c5d8 2026-02-20T18:02:29.830 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 7627652f-5ff6-4df6-868c-ab3170a7c5d8' 2026-02-20T18:02:29.830 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:02:29.838 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC1oZhp5OzdMRAAViT5moiToapO6Qe3atvreg== 2026-02-20T18:02:29.838 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC1oZhp5OzdMRAAViT5moiToapO6Qe3atvreg=="}' 2026-02-20T18:02:29.838 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 7627652f-5ff6-4df6-868c-ab3170a7c5d8 -i td/test-erasure-eio/6/new.json 2026-02-20T18:02:30.066 INFO:tasks.workunit.client.0.trial195.stdout:6 2026-02-20T18:02:30.074 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-20T18:02:30.075 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC1oZhp5OzdMRAAViT5moiToapO6Qe3atvreg== --osd-uuid 7627652f-5ff6-4df6-868c-ab3170a7c5d8 2026-02-20T18:02:30.093 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:30.091+0000 7f88dfc0e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:30.094 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:30.092+0000 7f88dfc0e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:30.095 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:30.093+0000 7f88dfc0e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:30.097 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:30.094+0000 7f88dfc0e900 -1 bdev(0x56162188ec00 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-20T18:02:30.097 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:30.094+0000 7f88dfc0e900 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-20T18:02:30.422 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-20T18:02:30.422 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:02:30.422 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-20T18:02:30.423 INFO:tasks.workunit.client.0.trial195.stdout:adding osd6 key to auth repository 2026-02-20T18:02:30.423 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:02:30.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-20T18:02:30.744 INFO:tasks.workunit.client.0.trial195.stdout:start osd.6 2026-02-20T18:02:30.745 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:02:30.745 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:02:30.745 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:02:30.745 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:02:30.761 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:30.759+0000 7f544b622900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:30.762 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:30.760+0000 7f544b622900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:30.763 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:30.761+0000 7f544b622900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:30.892 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:30.890+0000 7f544b622900 -1 Falling back to public interface 2026-02-20T18:02:30.970 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-20T18:02:30.970 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:02:30.970 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-20T18:02:30.970 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:02:30.970 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:02:30.971 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:02:30.971 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:02:30.971 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:02:30.972 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:02:30.972 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-20T18:02:31.194 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:02:31.230 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:31.228+0000 7f544b622900 -1 osd.6 0 log_to_monitors true 2026-02-20T18:02:32.195 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:02:32.195 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:02:32.196 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:02:32.196 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:02:32.197 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:02:32.197 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-20T18:02:32.419 INFO:tasks.workunit.client.0.trial195.stdout:osd.6 up in weight 1 up_from 52 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/3062852592,v1:127.0.0.1:6851/3062852592] [v2:127.0.0.1:6852/3062852592,v1:127.0.0.1:6853/3062852592] exists,up 7627652f-5ff6-4df6-868c-ab3170a7c5d8 2026-02-20T18:02:32.425 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:02:32.425 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:02:32.425 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:02:32.425 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:02:32.425 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 7 2026-02-20T18:02:32.425 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:02:32.425 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:02:32.426 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=7 2026-02-20T18:02:32.426 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:02:32.426 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/7 2026-02-20T18:02:32.426 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:02:32.426 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:02:32.426 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:02:32.426 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:02:32.426 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/7' 2026-02-20T18:02:32.426 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/7/journal' 2026-02-20T18:02:32.426 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:02:32.427 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:02:32.427 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:02:32.427 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:02:32.427 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:02:32.427 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:02:32.427 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:02:32.427 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:02:32.427 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:02:32.427 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:02:32.427 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:02:32.428 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:02:32.428 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:02:32.428 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:02:32.428 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:02:32.428 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:02:32.428 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:02:32.428 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:02:32.429 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:02:32.429 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:02:32.429 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:02:32.429 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:02:32.429 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/7 2026-02-20T18:02:32.429 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:02:32.429 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=3a8c19f0-f92d-490b-a3a8-b80b0f820759 2026-02-20T18:02:32.429 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd7 3a8c19f0-f92d-490b-a3a8-b80b0f820759' 2026-02-20T18:02:32.430 INFO:tasks.workunit.client.0.trial195.stdout:add osd7 3a8c19f0-f92d-490b-a3a8-b80b0f820759 2026-02-20T18:02:32.430 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:02:32.436 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC4oZhpZkLtGRAAVmD0DiW1hHmX2f7y6M8psw== 2026-02-20T18:02:32.436 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC4oZhpZkLtGRAAVmD0DiW1hHmX2f7y6M8psw=="}' 2026-02-20T18:02:32.436 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 3a8c19f0-f92d-490b-a3a8-b80b0f820759 -i td/test-erasure-eio/7/new.json 2026-02-20T18:02:32.667 INFO:tasks.workunit.client.0.trial195.stdout:7 2026-02-20T18:02:32.674 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/7/new.json 2026-02-20T18:02:32.675 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 7 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/7 --osd-journal=td/test-erasure-eio/7/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC4oZhpZkLtGRAAVmD0DiW1hHmX2f7y6M8psw== --osd-uuid 3a8c19f0-f92d-490b-a3a8-b80b0f820759 2026-02-20T18:02:32.694 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:32.693+0000 7f85338ba900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:32.696 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:32.694+0000 7f85338ba900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:32.697 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:32.695+0000 7f85338ba900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:32.697 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:32.695+0000 7f85338ba900 -1 bdev(0x55b957602c00 td/test-erasure-eio/7/block) open stat got: (1) Operation not permitted 2026-02-20T18:02:32.697 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:32.695+0000 7f85338ba900 -1 bluestore(td/test-erasure-eio/7) _read_fsid unparsable uuid 2026-02-20T18:02:33.015 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/7/keyring 2026-02-20T18:02:33.015 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:02:33.016 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd7 key to auth repository 2026-02-20T18:02:33.017 INFO:tasks.workunit.client.0.trial195.stdout:adding osd7 key to auth repository 2026-02-20T18:02:33.017 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/7/keyring auth add osd.7 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:02:33.331 INFO:tasks.workunit.client.0.trial195.stdout:start osd.7 2026-02-20T18:02:33.332 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.7 2026-02-20T18:02:33.332 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 7 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/7 --osd-journal=td/test-erasure-eio/7/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:02:33.332 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:02:33.332 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:02:33.332 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:02:33.348 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:33.346+0000 7f8ea7a0c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:33.350 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:33.348+0000 7f8ea7a0c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:33.350 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:33.349+0000 7f8ea7a0c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:33.486 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:33.484+0000 7f8ea7a0c900 -1 Falling back to public interface 2026-02-20T18:02:33.558 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 7 2026-02-20T18:02:33.559 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:02:33.559 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=7 2026-02-20T18:02:33.559 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:02:33.559 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:02:33.559 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:02:33.559 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:02:33.559 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:02:33.560 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:02:33.560 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-20T18:02:33.786 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:02:33.811 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:33.809+0000 7f8ea7a0c900 -1 osd.7 0 log_to_monitors true 2026-02-20T18:02:34.788 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:02:34.788 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:02:34.789 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:02:34.790 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:02:34.790 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:02:34.790 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-20T18:02:35.013 INFO:tasks.workunit.client.0.trial195.stdout:osd.7 up in weight 1 up_from 59 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6858/4126061670,v1:127.0.0.1:6859/4126061670] [v2:127.0.0.1:6860/4126061670,v1:127.0.0.1:6861/4126061670] exists,up 3a8c19f0-f92d-490b-a3a8-b80b0f820759 2026-02-20T18:02:35.014 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:02:35.014 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:02:35.014 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:02:35.014 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:02:35.014 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 8 2026-02-20T18:02:35.014 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:02:35.014 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:02:35.014 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=8 2026-02-20T18:02:35.014 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:02:35.014 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/8 2026-02-20T18:02:35.014 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:02:35.014 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:02:35.014 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:02:35.014 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:02:35.014 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/8' 2026-02-20T18:02:35.014 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/8/journal' 2026-02-20T18:02:35.014 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:02:35.014 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:02:35.014 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:02:35.014 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:02:35.014 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:02:35.015 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:02:35.015 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:02:35.015 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:02:35.015 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:02:35.015 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:02:35.016 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:02:35.016 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:02:35.016 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:02:35.016 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:02:35.016 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:02:35.016 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:02:35.016 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:02:35.016 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:02:35.016 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:02:35.017 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:02:35.017 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:02:35.017 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:02:35.017 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/8 2026-02-20T18:02:35.017 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:02:35.017 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=81b6ee48-c863-4b40-8b26-bd0800c65c9b 2026-02-20T18:02:35.018 INFO:tasks.workunit.client.0.trial195.stdout:add osd8 81b6ee48-c863-4b40-8b26-bd0800c65c9b 2026-02-20T18:02:35.018 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd8 81b6ee48-c863-4b40-8b26-bd0800c65c9b' 2026-02-20T18:02:35.019 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:02:35.029 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC7oZhpOLuoARAALF8lEGqfh+Xl2VWcn2oCVQ== 2026-02-20T18:02:35.029 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC7oZhpOLuoARAALF8lEGqfh+Xl2VWcn2oCVQ=="}' 2026-02-20T18:02:35.029 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 81b6ee48-c863-4b40-8b26-bd0800c65c9b -i td/test-erasure-eio/8/new.json 2026-02-20T18:02:35.256 INFO:tasks.workunit.client.0.trial195.stdout:8 2026-02-20T18:02:35.265 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/8/new.json 2026-02-20T18:02:35.266 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 8 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/8 --osd-journal=td/test-erasure-eio/8/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC7oZhpOLuoARAALF8lEGqfh+Xl2VWcn2oCVQ== --osd-uuid 81b6ee48-c863-4b40-8b26-bd0800c65c9b 2026-02-20T18:02:35.284 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:35.282+0000 7fb0278e8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:35.285 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:35.283+0000 7fb0278e8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:35.286 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:35.284+0000 7fb0278e8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:35.286 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:35.284+0000 7fb0278e8900 -1 bdev(0x55ef88004c00 td/test-erasure-eio/8/block) open stat got: (1) Operation not permitted 2026-02-20T18:02:35.287 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:35.284+0000 7fb0278e8900 -1 bluestore(td/test-erasure-eio/8) _read_fsid unparsable uuid 2026-02-20T18:02:35.605 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/8/keyring 2026-02-20T18:02:35.605 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:02:35.606 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd8 key to auth repository 2026-02-20T18:02:35.607 INFO:tasks.workunit.client.0.trial195.stdout:adding osd8 key to auth repository 2026-02-20T18:02:35.607 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/8/keyring auth add osd.8 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:02:35.936 INFO:tasks.workunit.client.0.trial195.stdout:start osd.8 2026-02-20T18:02:35.938 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.8 2026-02-20T18:02:35.938 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 8 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/8 --osd-journal=td/test-erasure-eio/8/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:02:35.938 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:02:35.939 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:02:35.939 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:02:35.953 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:35.951+0000 7f43dcc0f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:35.955 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:35.953+0000 7f43dcc0f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:35.955 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:35.954+0000 7f43dcc0f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:36.089 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:36.087+0000 7f43dcc0f900 -1 Falling back to public interface 2026-02-20T18:02:36.160 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 8 2026-02-20T18:02:36.161 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:02:36.161 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=8 2026-02-20T18:02:36.161 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:02:36.161 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:02:36.161 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:02:36.161 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:02:36.162 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:02:36.162 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:02:36.163 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-20T18:02:36.390 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:02:36.456 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:36.454+0000 7f43dcc0f900 -1 osd.8 0 log_to_monitors true 2026-02-20T18:02:37.391 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:02:37.392 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:02:37.392 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:02:37.393 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:02:37.393 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:02:37.393 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-20T18:02:37.621 INFO:tasks.workunit.client.0.trial195.stdout:osd.8 up in weight 1 up_from 67 up_thru 69 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6866/4193829413,v1:127.0.0.1:6867/4193829413] [v2:127.0.0.1:6868/4193829413,v1:127.0.0.1:6869/4193829413] exists,up 81b6ee48-c863-4b40-8b26-bd0800c65c9b 2026-02-20T18:02:37.622 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:02:37.622 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:02:37.623 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:02:37.623 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-20T18:02:37.623 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-20T18:02:37.623 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-20T18:02:37.623 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:02:37.623 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:02:37.623 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:02:37.623 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.0.asok 2026-02-20T18:02:37.623 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-20T18:02:37.623 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-osd.0.asok log flush 2026-02-20T18:02:37.668 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T18:02:37.671 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-20T18:02:37.674 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T18:02:15.665+0000 7f1ce1497900 0 load: jerasure load: lrc 2026-02-20T18:02:37.675 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:443: TEST_ec_recovery_multiple_errors: local poolname=pool-jerasure 2026-02-20T18:02:37.675 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:444: TEST_ec_recovery_multiple_errors: create_erasure_coded_pool pool-jerasure 4 4 2026-02-20T18:02:37.675 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-20T18:02:37.676 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-20T18:02:37.676 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=4 2026-02-20T18:02:37.676 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-20T18:02:37.676 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=4 2026-02-20T18:02:37.676 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-20T18:02:37.676 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=4 m=4 crush-failure-domain=osd 2026-02-20T18:02:37.960 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-20T18:02:37.961 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-20T18:02:38.331 INFO:tasks.workunit.client.0.trial195.stderr:pool 'pool-jerasure' already exists 2026-02-20T18:02:38.340 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-20T18:02:39.342 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-20T18:02:39.342 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:02:39.342 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:02:39.342 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:02:39.343 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:02:39.343 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:02:39.343 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:02:39.343 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:02:39.343 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:02:39.343 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:02:39.429 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:02:39.429 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:02:39.429 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:02:39.430 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:02:39.430 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:02:39.430 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:02:39.655 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:02:39.655 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:02:39.655 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:02:39.656 INFO:tasks.workunit.client.0.trial195.stderr:3 2026-02-20T18:02:39.656 INFO:tasks.workunit.client.0.trial195.stderr:4 2026-02-20T18:02:39.656 INFO:tasks.workunit.client.0.trial195.stderr:5 2026-02-20T18:02:39.656 INFO:tasks.workunit.client.0.trial195.stderr:6 2026-02-20T18:02:39.656 INFO:tasks.workunit.client.0.trial195.stderr:7 2026-02-20T18:02:39.656 INFO:tasks.workunit.client.0.trial195.stderr:8' 2026-02-20T18:02:39.656 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:02:39.656 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:02:39.656 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:02:39.720 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803783 2026-02-20T18:02:39.720 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803783 2026-02-20T18:02:39.720 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803783' 2026-02-20T18:02:39.721 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:02:39.721 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:02:39.784 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574855 2026-02-20T18:02:39.784 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574855 2026-02-20T18:02:39.784 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803783 1-55834574855' 2026-02-20T18:02:39.784 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:02:39.784 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:02:39.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411334 2026-02-20T18:02:39.849 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411334 2026-02-20T18:02:39.849 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803783 1-55834574855 2-77309411334' 2026-02-20T18:02:39.849 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:02:39.849 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:02:39.911 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964116998 2026-02-20T18:02:39.911 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964116998 2026-02-20T18:02:39.912 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803783 1-55834574855 2-77309411334 3-115964116998' 2026-02-20T18:02:39.912 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:02:39.912 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-20T18:02:39.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=154618822661 2026-02-20T18:02:39.976 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 154618822661 2026-02-20T18:02:39.976 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803783 1-55834574855 2-77309411334 3-115964116998 4-154618822661' 2026-02-20T18:02:39.976 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:02:39.976 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-20T18:02:40.039 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=188978561029 2026-02-20T18:02:40.040 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 188978561029 2026-02-20T18:02:40.040 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803783 1-55834574855 2-77309411334 3-115964116998 4-154618822661 5-188978561029' 2026-02-20T18:02:40.040 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:02:40.040 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-20T18:02:40.104 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=223338299396 2026-02-20T18:02:40.104 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 223338299396 2026-02-20T18:02:40.105 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803783 1-55834574855 2-77309411334 3-115964116998 4-154618822661 5-188978561029 6-223338299396' 2026-02-20T18:02:40.105 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:02:40.105 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-20T18:02:40.169 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=253403070468 2026-02-20T18:02:40.169 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 253403070468 2026-02-20T18:02:40.169 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803783 1-55834574855 2-77309411334 3-115964116998 4-154618822661 5-188978561029 6-223338299396 7-253403070468' 2026-02-20T18:02:40.169 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:02:40.169 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-20T18:02:40.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=287762808835 2026-02-20T18:02:40.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 287762808835 2026-02-20T18:02:40.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803783 1-55834574855 2-77309411334 3-115964116998 4-154618822661 5-188978561029 6-223338299396 7-253403070468 8-287762808835' 2026-02-20T18:02:40.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:02:40.234 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803783 2026-02-20T18:02:40.235 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:02:40.235 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:02:40.235 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803783 2026-02-20T18:02:40.236 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:02:40.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803783 2026-02-20T18:02:40.237 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.0 seq 25769803783 2026-02-20T18:02:40.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803783' 2026-02-20T18:02:40.237 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:02:40.464 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803781 -lt 25769803783 2026-02-20T18:02:40.464 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:02:41.466 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:02:41.466 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:02:41.695 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803783 -lt 25769803783 2026-02-20T18:02:41.695 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:02:41.695 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574855 2026-02-20T18:02:41.695 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:02:41.696 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:02:41.696 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-55834574855 2026-02-20T18:02:41.696 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:02:41.697 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574855 2026-02-20T18:02:41.697 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 55834574855' 2026-02-20T18:02:41.697 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.1 seq 55834574855 2026-02-20T18:02:41.698 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:02:41.921 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574855 -lt 55834574855 2026-02-20T18:02:41.921 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:02:41.921 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411334 2026-02-20T18:02:41.921 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:02:41.922 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:02:41.923 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-77309411334 2026-02-20T18:02:41.923 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:02:41.924 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=77309411334 2026-02-20T18:02:41.924 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 77309411334' 2026-02-20T18:02:41.924 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.2 seq 77309411334 2026-02-20T18:02:41.924 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:02:42.147 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411334 -lt 77309411334 2026-02-20T18:02:42.148 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:02:42.148 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964116998 2026-02-20T18:02:42.148 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:02:42.149 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:02:42.149 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-115964116998 2026-02-20T18:02:42.149 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:02:42.151 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=115964116998 2026-02-20T18:02:42.151 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 115964116998' 2026-02-20T18:02:42.151 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.3 seq 115964116998 2026-02-20T18:02:42.151 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:02:42.380 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 115964116998 -lt 115964116998 2026-02-20T18:02:42.380 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:02:42.381 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-154618822661 2026-02-20T18:02:42.381 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:02:42.382 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-20T18:02:42.382 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-154618822661 2026-02-20T18:02:42.383 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:02:42.383 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=154618822661 2026-02-20T18:02:42.384 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 154618822661' 2026-02-20T18:02:42.384 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.4 seq 154618822661 2026-02-20T18:02:42.384 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-20T18:02:42.611 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 154618822661 -lt 154618822661 2026-02-20T18:02:42.611 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:02:42.612 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-188978561029 2026-02-20T18:02:42.612 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:02:42.612 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-20T18:02:42.613 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-188978561029 2026-02-20T18:02:42.613 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:02:42.614 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=188978561029 2026-02-20T18:02:42.614 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 188978561029' 2026-02-20T18:02:42.614 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.5 seq 188978561029 2026-02-20T18:02:42.614 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-20T18:02:42.841 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 188978561029 -lt 188978561029 2026-02-20T18:02:42.841 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:02:42.841 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-223338299396 2026-02-20T18:02:42.842 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:02:42.842 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-20T18:02:42.843 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-223338299396 2026-02-20T18:02:42.843 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:02:42.844 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=223338299396 2026-02-20T18:02:42.844 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 223338299396' 2026-02-20T18:02:42.845 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.6 seq 223338299396 2026-02-20T18:02:42.845 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-20T18:02:43.072 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 223338299396 -lt 223338299396 2026-02-20T18:02:43.072 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:02:43.073 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-253403070468 2026-02-20T18:02:43.073 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:02:43.073 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-20T18:02:43.074 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-253403070468 2026-02-20T18:02:43.074 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:02:43.075 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=253403070468 2026-02-20T18:02:43.075 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 253403070468' 2026-02-20T18:02:43.076 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.7 seq 253403070468 2026-02-20T18:02:43.076 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-20T18:02:43.302 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 253403070468 -lt 253403070468 2026-02-20T18:02:43.302 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:02:43.303 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-287762808835 2026-02-20T18:02:43.303 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:02:43.304 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-20T18:02:43.304 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-287762808835 2026-02-20T18:02:43.304 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:02:43.305 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=287762808835 2026-02-20T18:02:43.305 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 287762808835' 2026-02-20T18:02:43.305 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.8 seq 287762808835 2026-02-20T18:02:43.306 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-20T18:02:43.534 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 287762808835 -lt 287762808835 2026-02-20T18:02:43.535 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:02:43.535 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:02:43.535 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:02:43.846 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:02:43.846 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:02:43.847 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:02:43.847 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:02:43.847 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:02:43.847 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:02:43.847 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:02:43.847 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:02:44.068 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:02:44.069 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:02:44.069 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:02:44.069 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:02:44.386 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:02:44.386 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:02:44.387 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:02:44.387 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:446: TEST_ec_recovery_multiple_errors: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-20T18:02:44.387 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-20T18:02:44.387 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-20T18:02:44.387 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-20T18:02:44.387 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:02:44.387 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-20T18:02:44.387 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:02:44.387 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-20T18:02:44.387 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:02:44.388 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-20T18:02:44.388 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:02:44.388 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-20T18:02:44.388 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-20T18:02:44.408 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:447: TEST_ec_recovery_multiple_errors: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 0 2026-02-20T18:02:44.408 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=ec 2026-02-20T18:02:44.408 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-20T18:02:44.408 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=data 2026-02-20T18:02:44.408 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-20T18:02:44.408 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=pool-jerasure 2026-02-20T18:02:44.408 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-20T18:02:44.408 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=myobject 2026-02-20T18:02:44.408 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-20T18:02:44.409 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/test-erasure-eio 2026-02-20T18:02:44.409 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-20T18:02:44.409 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=0 2026-02-20T18:02:44.409 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-20T18:02:44.409 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds pool-jerasure myobject 2026-02-20T18:02:44.409 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:02:44.409 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-20T18:02:44.409 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-20T18:02:44.409 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:02:44.642 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T18:02:44.642 INFO:tasks.workunit.client.0.trial195.stderr:5 2026-02-20T18:02:44.642 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:02:44.642 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T18:02:44.643 INFO:tasks.workunit.client.0.trial195.stderr:6 2026-02-20T18:02:44.643 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:02:44.643 INFO:tasks.workunit.client.0.trial195.stderr:4 2026-02-20T18:02:44.643 INFO:tasks.workunit.client.0.trial195.stderr:7' 2026-02-20T18:02:44.643 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2 4 7 2026-02-20T18:02:44.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('3' '5' '1' '0' '6' '2' '4' '7') 2026-02-20T18:02:44.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-20T18:02:44.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=3 2026-02-20T18:02:44.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' ec '!=' ec ']' 2026-02-20T18:02:44.643 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/test-erasure-eio/3/type 2026-02-20T18:02:44.645 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-20T18:02:44.645 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-20T18:02:44.645 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-20T18:02:44.645 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-20T18:02:44.646 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-20T18:02:44.646 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-20T18:02:44.646 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-20T18:02:44.646 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-20T18:02:44.646 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-20T18:02:44.646 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-20T18:02:44.646 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:02:44.646 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:02:44.646 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:02:44.646 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.3.asok 2026-02-20T18:02:44.646 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.85808/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-20T18:02:44.694 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-20T18:02:44.694 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-20T18:02:44.695 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-20T18:02:44.695 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.3 2026-02-20T18:02:44.696 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-20T18:02:44.696 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-20T18:02:44.696 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:02:44.696 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:02:44.696 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:02:44.696 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.3.asok 2026-02-20T18:02:44.696 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-20T18:02:44.696 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-osd.3.asok injectdataerr pool-jerasure myobject 0 2026-02-20T18:02:44.745 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:450: TEST_ec_recovery_multiple_errors: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 3 2026-02-20T18:02:44.746 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=ec 2026-02-20T18:02:44.746 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-20T18:02:44.746 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=data 2026-02-20T18:02:44.746 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-20T18:02:44.746 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=pool-jerasure 2026-02-20T18:02:44.746 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-20T18:02:44.746 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=myobject 2026-02-20T18:02:44.746 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-20T18:02:44.746 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/test-erasure-eio 2026-02-20T18:02:44.746 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-20T18:02:44.747 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=3 2026-02-20T18:02:44.747 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-20T18:02:44.747 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds pool-jerasure myobject 2026-02-20T18:02:44.747 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:02:44.747 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-20T18:02:44.747 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-20T18:02:44.747 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:02:44.975 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T18:02:44.975 INFO:tasks.workunit.client.0.trial195.stderr:5 2026-02-20T18:02:44.975 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:02:44.975 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T18:02:44.975 INFO:tasks.workunit.client.0.trial195.stderr:6 2026-02-20T18:02:44.976 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:02:44.976 INFO:tasks.workunit.client.0.trial195.stderr:4 2026-02-20T18:02:44.976 INFO:tasks.workunit.client.0.trial195.stderr:7' 2026-02-20T18:02:44.976 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2 4 7 2026-02-20T18:02:44.976 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('3' '5' '1' '0' '6' '2' '4' '7') 2026-02-20T18:02:44.976 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-20T18:02:44.976 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=0 2026-02-20T18:02:44.976 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' ec '!=' ec ']' 2026-02-20T18:02:44.976 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/test-erasure-eio/0/type 2026-02-20T18:02:44.978 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-20T18:02:44.978 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 0 bluestore_debug_inject_read_err true 2026-02-20T18:02:44.978 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-20T18:02:44.979 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=0 2026-02-20T18:02:44.979 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-20T18:02:44.979 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-20T18:02:44.979 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-20T18:02:44.979 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.0 2026-02-20T18:02:44.979 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-20T18:02:44.979 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-20T18:02:44.979 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:02:44.979 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:02:44.979 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:02:44.979 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.0.asok 2026-02-20T18:02:44.979 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.85808/ceph-osd.0.asok config set bluestore_debug_inject_read_err true 2026-02-20T18:02:45.029 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-20T18:02:45.029 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-20T18:02:45.031 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-20T18:02:45.031 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.0 2026-02-20T18:02:45.031 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-20T18:02:45.031 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-20T18:02:45.031 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:02:45.031 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:02:45.032 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:02:45.032 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.0.asok 2026-02-20T18:02:45.032 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-20T18:02:45.032 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-osd.0.asok injectdataerr pool-jerasure myobject 3 2026-02-20T18:02:45.080 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:451: TEST_ec_recovery_multiple_errors: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 4 2026-02-20T18:02:45.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=ec 2026-02-20T18:02:45.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-20T18:02:45.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=data 2026-02-20T18:02:45.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-20T18:02:45.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=pool-jerasure 2026-02-20T18:02:45.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-20T18:02:45.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=myobject 2026-02-20T18:02:45.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-20T18:02:45.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/test-erasure-eio 2026-02-20T18:02:45.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-20T18:02:45.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=4 2026-02-20T18:02:45.082 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-20T18:02:45.082 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds pool-jerasure myobject 2026-02-20T18:02:45.082 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:02:45.082 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-20T18:02:45.082 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-20T18:02:45.082 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:02:45.305 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T18:02:45.305 INFO:tasks.workunit.client.0.trial195.stderr:5 2026-02-20T18:02:45.305 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:02:45.306 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T18:02:45.306 INFO:tasks.workunit.client.0.trial195.stderr:6 2026-02-20T18:02:45.306 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:02:45.306 INFO:tasks.workunit.client.0.trial195.stderr:4 2026-02-20T18:02:45.306 INFO:tasks.workunit.client.0.trial195.stderr:7' 2026-02-20T18:02:45.306 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2 4 7 2026-02-20T18:02:45.306 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('3' '5' '1' '0' '6' '2' '4' '7') 2026-02-20T18:02:45.306 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-20T18:02:45.306 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=6 2026-02-20T18:02:45.306 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' ec '!=' ec ']' 2026-02-20T18:02:45.306 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/test-erasure-eio/6/type 2026-02-20T18:02:45.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-20T18:02:45.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 6 bluestore_debug_inject_read_err true 2026-02-20T18:02:45.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-20T18:02:45.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=6 2026-02-20T18:02:45.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-20T18:02:45.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-20T18:02:45.309 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-20T18:02:45.309 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.6 2026-02-20T18:02:45.309 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.6 2026-02-20T18:02:45.309 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.6 ']' 2026-02-20T18:02:45.309 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:02:45.309 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:02:45.309 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:02:45.309 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.6.asok 2026-02-20T18:02:45.310 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.85808/ceph-osd.6.asok config set bluestore_debug_inject_read_err true 2026-02-20T18:02:45.356 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-20T18:02:45.356 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-20T18:02:45.357 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-20T18:02:45.357 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.6 2026-02-20T18:02:45.357 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.6 2026-02-20T18:02:45.357 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.6 ']' 2026-02-20T18:02:45.358 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:02:45.358 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:02:45.358 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:02:45.358 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.6.asok 2026-02-20T18:02:45.358 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-20T18:02:45.358 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-osd.6.asok injectdataerr pool-jerasure myobject 4 2026-02-20T18:02:45.408 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:453: TEST_ec_recovery_multiple_errors: get_osds pool-jerasure myobject 2026-02-20T18:02:45.408 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:02:45.408 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-20T18:02:45.409 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-20T18:02:45.409 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:02:45.636 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T18:02:45.636 INFO:tasks.workunit.client.0.trial195.stderr:5 2026-02-20T18:02:45.636 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:02:45.636 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T18:02:45.636 INFO:tasks.workunit.client.0.trial195.stderr:6 2026-02-20T18:02:45.636 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:02:45.636 INFO:tasks.workunit.client.0.trial195.stderr:4 2026-02-20T18:02:45.636 INFO:tasks.workunit.client.0.trial195.stderr:7' 2026-02-20T18:02:45.636 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2 4 7 2026-02-20T18:02:45.636 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:453: TEST_ec_recovery_multiple_errors: initial_osds=('3' '5' '1' '0' '6' '2' '4' '7') 2026-02-20T18:02:45.636 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:453: TEST_ec_recovery_multiple_errors: local -a initial_osds 2026-02-20T18:02:45.636 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:454: TEST_ec_recovery_multiple_errors: local last_osd=7 2026-02-20T18:02:45.636 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:456: TEST_ec_recovery_multiple_errors: kill_daemons td/test-erasure-eio TERM osd.7 2026-02-20T18:02:45.637 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:02:45.637 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:02:45.637 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:02:45.637 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:02:45.637 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:02:45.944 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:02:45.944 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:457: TEST_ec_recovery_multiple_errors: ceph osd down 7 2026-02-20T18:02:46.168 INFO:tasks.workunit.client.0.trial195.stderr:osd.7 is already down. 2026-02-20T18:02:46.176 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:458: TEST_ec_recovery_multiple_errors: ceph osd out 7 2026-02-20T18:02:46.421 INFO:tasks.workunit.client.0.trial195.stderr:osd.7 is already out. 2026-02-20T18:02:46.430 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:461: TEST_ec_recovery_multiple_errors: wait_for_clean 2026-02-20T18:02:46.430 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:02:46.430 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:02:46.430 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:02:46.430 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:02:46.430 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:02:46.430 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:02:46.431 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:02:46.431 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:02:46.431 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:02:46.516 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:02:46.516 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:02:46.516 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:02:46.516 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:02:46.516 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:02:46.516 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:02:46.745 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:02:46.745 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:02:46.745 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:02:46.745 INFO:tasks.workunit.client.0.trial195.stderr:3 2026-02-20T18:02:46.746 INFO:tasks.workunit.client.0.trial195.stderr:4 2026-02-20T18:02:46.746 INFO:tasks.workunit.client.0.trial195.stderr:5 2026-02-20T18:02:46.746 INFO:tasks.workunit.client.0.trial195.stderr:6 2026-02-20T18:02:46.746 INFO:tasks.workunit.client.0.trial195.stderr:7 2026-02-20T18:02:46.746 INFO:tasks.workunit.client.0.trial195.stderr:8' 2026-02-20T18:02:46.746 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:02:46.746 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:02:46.746 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:02:46.810 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803786 2026-02-20T18:02:46.810 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803786 2026-02-20T18:02:46.810 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786' 2026-02-20T18:02:46.810 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:02:46.810 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:02:46.874 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574858 2026-02-20T18:02:46.875 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574858 2026-02-20T18:02:46.875 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786 1-55834574858' 2026-02-20T18:02:46.875 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:02:46.875 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:02:46.937 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411337 2026-02-20T18:02:46.937 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411337 2026-02-20T18:02:46.938 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786 1-55834574858 2-77309411337' 2026-02-20T18:02:46.938 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:02:46.938 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:02:47.005 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964117001 2026-02-20T18:02:47.005 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964117001 2026-02-20T18:02:47.005 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786 1-55834574858 2-77309411337 3-115964117001' 2026-02-20T18:02:47.005 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:02:47.006 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-20T18:02:47.069 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=154618822664 2026-02-20T18:02:47.069 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 154618822664 2026-02-20T18:02:47.069 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786 1-55834574858 2-77309411337 3-115964117001 4-154618822664' 2026-02-20T18:02:47.069 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:02:47.070 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-20T18:02:47.134 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=188978561032 2026-02-20T18:02:47.134 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 188978561032 2026-02-20T18:02:47.134 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786 1-55834574858 2-77309411337 3-115964117001 4-154618822664 5-188978561032' 2026-02-20T18:02:47.134 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:02:47.134 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-20T18:02:47.198 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=223338299399 2026-02-20T18:02:47.198 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 223338299399 2026-02-20T18:02:47.198 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786 1-55834574858 2-77309411337 3-115964117001 4-154618822664 5-188978561032 6-223338299399' 2026-02-20T18:02:47.198 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:02:47.199 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-20T18:02:47.243 INFO:tasks.workunit.client.0.trial195.stderr:Error ENXIO: problem getting command descriptions from osd.7 2026-02-20T18:02:47.244 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq= 2026-02-20T18:02:47.245 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z '' 2026-02-20T18:02:47.245 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2261: flush_pg_stats: continue 2026-02-20T18:02:47.245 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:02:47.245 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-20T18:02:47.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=287762808838 2026-02-20T18:02:47.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 287762808838 2026-02-20T18:02:47.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786 1-55834574858 2-77309411337 3-115964117001 4-154618822664 5-188978561032 6-223338299399 8-287762808838' 2026-02-20T18:02:47.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:02:47.308 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803786 2026-02-20T18:02:47.308 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:02:47.310 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:02:47.310 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803786 2026-02-20T18:02:47.310 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:02:47.311 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803786 2026-02-20T18:02:47.311 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803786' 2026-02-20T18:02:47.311 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.0 seq 25769803786 2026-02-20T18:02:47.311 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:02:47.549 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803784 -lt 25769803786 2026-02-20T18:02:47.549 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:02:48.550 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:02:48.550 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:02:48.772 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803787 -lt 25769803786 2026-02-20T18:02:48.772 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:02:48.773 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574858 2026-02-20T18:02:48.773 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:02:48.773 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:02:48.774 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-55834574858 2026-02-20T18:02:48.774 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:02:48.775 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574858 2026-02-20T18:02:48.775 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 55834574858' 2026-02-20T18:02:48.775 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.1 seq 55834574858 2026-02-20T18:02:48.776 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:02:48.999 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574858 -lt 55834574858 2026-02-20T18:02:48.999 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:02:49.000 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411337 2026-02-20T18:02:49.000 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:02:49.000 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:02:49.001 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-77309411337 2026-02-20T18:02:49.001 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:02:49.002 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=77309411337 2026-02-20T18:02:49.002 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 77309411337' 2026-02-20T18:02:49.003 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.2 seq 77309411337 2026-02-20T18:02:49.003 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:02:49.227 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411338 -lt 77309411337 2026-02-20T18:02:49.227 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:02:49.228 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964117001 2026-02-20T18:02:49.228 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:02:49.228 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:02:49.229 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-115964117001 2026-02-20T18:02:49.229 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:02:49.230 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=115964117001 2026-02-20T18:02:49.230 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 115964117001' 2026-02-20T18:02:49.231 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.3 seq 115964117001 2026-02-20T18:02:49.231 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:02:49.455 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 115964117001 -lt 115964117001 2026-02-20T18:02:49.456 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:02:49.456 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-154618822664 2026-02-20T18:02:49.457 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:02:49.457 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-20T18:02:49.458 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-154618822664 2026-02-20T18:02:49.458 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:02:49.459 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=154618822664 2026-02-20T18:02:49.459 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 154618822664' 2026-02-20T18:02:49.459 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.4 seq 154618822664 2026-02-20T18:02:49.459 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-20T18:02:49.682 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 154618822665 -lt 154618822664 2026-02-20T18:02:49.682 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:02:49.682 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-188978561032 2026-02-20T18:02:49.683 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:02:49.683 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-20T18:02:49.684 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-188978561032 2026-02-20T18:02:49.684 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:02:49.685 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=188978561032 2026-02-20T18:02:49.686 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 188978561032' 2026-02-20T18:02:49.686 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.5 seq 188978561032 2026-02-20T18:02:49.686 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-20T18:02:49.914 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 188978561032 -lt 188978561032 2026-02-20T18:02:49.914 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:02:49.915 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-223338299399 2026-02-20T18:02:49.915 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:02:49.915 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-20T18:02:49.916 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-223338299399 2026-02-20T18:02:49.916 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:02:49.917 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=223338299399 2026-02-20T18:02:49.918 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.6 seq 223338299399 2026-02-20T18:02:49.918 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 223338299399' 2026-02-20T18:02:49.918 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-20T18:02:50.144 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 223338299400 -lt 223338299399 2026-02-20T18:02:50.144 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:02:50.145 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-287762808838 2026-02-20T18:02:50.145 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:02:50.146 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-20T18:02:50.146 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-287762808838 2026-02-20T18:02:50.146 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:02:50.147 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=287762808838 2026-02-20T18:02:50.148 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 287762808838' 2026-02-20T18:02:50.148 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.8 seq 287762808838 2026-02-20T18:02:50.148 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-20T18:02:50.375 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 287762808839 -lt 287762808838 2026-02-20T18:02:50.376 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:02:50.376 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:02:50.376 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:02:50.687 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:02:50.687 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:02:50.687 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:02:50.687 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:02:50.687 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:02:50.687 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:02:50.687 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:02:50.688 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:02:50.907 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:02:50.908 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:02:50.908 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:02:50.908 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:02:51.220 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:02:51.221 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:02:51.221 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:02:51.221 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:463: TEST_ec_recovery_multiple_errors: rados_get td/test-erasure-eio pool-jerasure myobject 2026-02-20T18:02:51.221 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-20T18:02:51.222 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-20T18:02:51.222 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=myobject 2026-02-20T18:02:51.222 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-20T18:02:51.222 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-20T18:02:51.222 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get myobject td/test-erasure-eio/COPY 2026-02-20T18:02:51.241 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-20T18:02:51.241 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-20T18:02:51.242 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:465: TEST_ec_recovery_multiple_errors: delete_erasure_coded_pool pool-jerasure 2026-02-20T18:02:51.242 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-20T18:02:51.242 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-20T18:02:51.511 INFO:tasks.workunit.client.0.trial195.stderr:pool 'pool-jerasure' does not exist 2026-02-20T18:02:51.519 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-20T18:02:51.819 INFO:tasks.workunit.client.0.trial195.stderr:erasure-code-profile myprofile does not exist 2026-02-20T18:02:51.827 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-20T18:02:51.827 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-20T18:02:51.827 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T18:02:51.828 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-20T18:02:51.828 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:02:51.828 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:02:51.829 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:02:51.829 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:02:51.829 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:02:51.940 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:02:51.940 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T18:02:51.941 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T18:02:51.941 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T18:02:51.942 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T18:02:51.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T18:02:51.943 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T18:02:51.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:02:51.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T18:02:51.944 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T18:02:51.944 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:02:51.945 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T18:02:51.946 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T18:02:51.946 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-20T18:02:51.963 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T18:02:51.963 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:02:51.963 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:02:51.963 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.85808 2026-02-20T18:02:51.965 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T18:02:51.965 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T18:02:51.965 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-20T18:02:51.965 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-20T18:02:51.966 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-20T18:02:51.966 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-20T18:02:51.966 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-20T18:02:51.966 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T18:02:51.966 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-20T18:02:51.966 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:02:51.966 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:02:51.966 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:02:51.966 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:02:51.966 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:02:51.967 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:02:51.967 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T18:02:51.967 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T18:02:51.968 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T18:02:51.969 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T18:02:51.969 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T18:02:51.969 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T18:02:51.970 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:02:51.970 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T18:02:51.970 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T18:02:51.970 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:02:51.971 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T18:02:51.972 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T18:02:51.972 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-20T18:02:51.973 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T18:02:51.973 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:02:51.973 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:02:51.973 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.85808 2026-02-20T18:02:51.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T18:02:51.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T18:02:51.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-20T18:02:51.975 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-20T18:02:51.975 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:02:51.976 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:02:51.976 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.85808 2026-02-20T18:02:51.976 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-20T18:02:51.978 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-20T18:02:51.978 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-20T18:02:51.978 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-20T18:02:51.978 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-20T18:02:51.978 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-20T18:02:51.978 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-20T18:02:51.978 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-20T18:02:51.978 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-20T18:02:51.978 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-20T18:02:51.978 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-20T18:02:51.999 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-20T18:02:51.999 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:02:52.000 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:02:52.000 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:02:52.000 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:02:52.000 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:02:52.000 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:02:52.000 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-20T18:02:52.026 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-20T18:02:52.027 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-20T18:02:52.027 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T18:02:52.027 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T18:02:52.027 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-20T18:02:52.027 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-20T18:02:52.027 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T18:02:52.027 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:02:52.028 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:02:52.028 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:02:52.029 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:02:52.029 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:02:52.029 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:02:52.029 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T18:02:52.029 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.85808/ceph-mon.a.asok config get fsid 2026-02-20T18:02:52.071 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-20T18:02:52.072 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T18:02:52.072 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T18:02:52.072 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-20T18:02:52.072 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-20T18:02:52.072 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T18:02:52.072 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:02:52.072 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:02:52.072 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:02:52.072 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:02:52.072 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:02:52.073 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:02:52.073 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T18:02:52.073 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.85808/ceph-mon.a.asok config get mon_host 2026-02-20T18:02:52.118 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-20T18:02:52.118 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-20T18:02:52.119 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-20T18:02:52.119 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-20T18:02:52.119 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-20T18:02:52.119 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-20T18:02:52.119 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-20T18:02:52.222 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-20T18:02:52.222 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:02:52.222 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:02:52.223 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:02:52.223 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:02:52.223 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:02:52.223 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:02:52.223 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T18:02:52.224 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T18:02:52.241 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-20T18:02:52.241 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-20T18:02:52.338 INFO:tasks.workunit.client.0.trial195.stderr:pool 'rbd' already exists 2026-02-20T18:02:52.344 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-20T18:02:53.346 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-20T18:02:53.346 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:02:53.346 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:02:53.346 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:02:53.346 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:02:53.346 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:02:53.346 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:02:53.346 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-20T18:02:53.346 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-mon.a.asok log flush 2026-02-20T18:02:53.388 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T18:02:53.393 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-20T18:02:53.394 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T18:02:52.019+0000 7fe533222d80 0 load: jerasure load: lrc 2026-02-20T18:02:53.395 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_recovery_multiple_objects td/test-erasure-eio 2026-02-20T18:02:53.395 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:471: TEST_ec_recovery_multiple_objects: local dir=td/test-erasure-eio 2026-02-20T18:02:53.395 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:472: TEST_ec_recovery_multiple_objects: local objname=myobject 2026-02-20T18:02:53.395 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:474: TEST_ec_recovery_multiple_objects: ORIG_ARGS='--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:02:53.395 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:475: TEST_ec_recovery_multiple_objects: CEPH_ARGS+=' --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-20T18:02:53.395 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:476: TEST_ec_recovery_multiple_objects: setup_osds 7 2026-02-20T18:02:53.395 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=7 2026-02-20T18:02:53.395 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-20T18:02:53.395 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 7 - 1 2026-02-20T18:02:53.396 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 6 2026-02-20T18:02:53.397 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:02:53.397 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-20T18:02:53.397 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:02:53.398 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:02:53.398 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-20T18:02:53.398 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:02:53.398 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-20T18:02:53.398 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-20T18:02:53.398 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:02:53.398 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:02:53.398 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:02:53.398 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-20T18:02:53.398 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-20T18:02:53.398 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:02:53.398 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:02:53.398 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:02:53.398 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:02:53.398 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:02:53.398 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:02:53.398 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:02:53.398 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:02:53.398 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:02:53.398 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:02:53.399 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:02:53.399 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:02:53.399 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:02:53.399 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:02:53.399 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:02:53.399 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:02:53.399 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:02:53.399 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:02:53.399 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:02:53.399 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:02:53.399 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:02:53.399 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:02:53.399 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-20T18:02:53.399 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:02:53.400 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=7990bde7-8c94-4d1c-b5ac-95b9dd8cd1e3 2026-02-20T18:02:53.400 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 7990bde7-8c94-4d1c-b5ac-95b9dd8cd1e3' 2026-02-20T18:02:53.400 INFO:tasks.workunit.client.0.trial195.stdout:add osd0 7990bde7-8c94-4d1c-b5ac-95b9dd8cd1e3 2026-02-20T18:02:53.400 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:02:53.414 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDNoZhpv6eaGBAA9DTwPdu2PV8rIMQgkPZfOQ== 2026-02-20T18:02:53.414 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDNoZhpv6eaGBAA9DTwPdu2PV8rIMQgkPZfOQ=="}' 2026-02-20T18:02:53.414 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 7990bde7-8c94-4d1c-b5ac-95b9dd8cd1e3 -i td/test-erasure-eio/0/new.json 2026-02-20T18:02:53.519 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:02:53.525 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-20T18:02:53.526 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDNoZhpv6eaGBAA9DTwPdu2PV8rIMQgkPZfOQ== --osd-uuid 7990bde7-8c94-4d1c-b5ac-95b9dd8cd1e3 2026-02-20T18:02:53.546 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:53.545+0000 7f52048d9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:53.548 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:53.547+0000 7f52048d9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:53.550 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:53.548+0000 7f52048d9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:53.550 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:53.549+0000 7f52048d9900 -1 bdev(0x5652c62bec00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-20T18:02:53.550 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:53.549+0000 7f52048d9900 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-20T18:02:53.967 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-20T18:02:53.968 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:02:53.968 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-20T18:02:53.969 INFO:tasks.workunit.client.0.trial195.stdout:adding osd0 key to auth repository 2026-02-20T18:02:53.969 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:02:54.075 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-20T18:02:54.076 INFO:tasks.workunit.client.0.trial195.stdout:start osd.0 2026-02-20T18:02:54.077 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:02:54.077 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:02:54.077 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:02:54.077 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:02:54.095 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:54.094+0000 7f9a665da900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:54.096 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:54.095+0000 7f9a665da900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:54.097 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:54.096+0000 7f9a665da900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:54.178 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-20T18:02:54.178 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:02:54.178 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-20T18:02:54.178 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:02:54.178 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:02:54.178 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:02:54.178 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:02:54.179 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:02:54.179 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:02:54.179 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T18:02:54.279 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:02:54.293 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:54.293+0000 7f9a665da900 -1 Falling back to public interface 2026-02-20T18:02:54.564 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:54.564+0000 7f9a665da900 -1 osd.0 0 log_to_monitors true 2026-02-20T18:02:55.281 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:02:55.281 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:02:55.281 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:02:55.282 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:02:55.282 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:02:55.282 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T18:02:55.507 INFO:tasks.workunit.client.0.trial195.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3961434956,v1:127.0.0.1:6803/3961434956] [v2:127.0.0.1:6804/3961434956,v1:127.0.0.1:6805/3961434956] exists,up 7990bde7-8c94-4d1c-b5ac-95b9dd8cd1e3 2026-02-20T18:02:55.512 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:02:55.512 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:02:55.512 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:02:55.512 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:02:55.512 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-20T18:02:55.512 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:02:55.512 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:02:55.513 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-20T18:02:55.513 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:02:55.513 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-20T18:02:55.513 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-20T18:02:55.513 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:02:55.513 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:02:55.513 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:02:55.513 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-20T18:02:55.513 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-20T18:02:55.513 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:02:55.513 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:02:55.514 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:02:55.514 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:02:55.514 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:02:55.514 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:02:55.514 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:02:55.514 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:02:55.514 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:02:55.514 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:02:55.514 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:02:55.514 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:02:55.514 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:02:55.514 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:02:55.515 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:02:55.515 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:02:55.515 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:02:55.515 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:02:55.515 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:02:55.515 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:02:55.515 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:02:55.515 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:02:55.515 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-20T18:02:55.515 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:02:55.515 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a10f71dd-3057-48c1-8275-ed584003e255 2026-02-20T18:02:55.515 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 a10f71dd-3057-48c1-8275-ed584003e255' 2026-02-20T18:02:55.516 INFO:tasks.workunit.client.0.trial195.stdout:add osd1 a10f71dd-3057-48c1-8275-ed584003e255 2026-02-20T18:02:55.516 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:02:55.525 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDPoZhpyhA9HxAAG0R4ORa/QGc20Tpz7okJ6w== 2026-02-20T18:02:55.525 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDPoZhpyhA9HxAAG0R4ORa/QGc20Tpz7okJ6w=="}' 2026-02-20T18:02:55.525 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a10f71dd-3057-48c1-8275-ed584003e255 -i td/test-erasure-eio/1/new.json 2026-02-20T18:02:55.756 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:02:55.764 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-20T18:02:55.765 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDPoZhpyhA9HxAAG0R4ORa/QGc20Tpz7okJ6w== --osd-uuid a10f71dd-3057-48c1-8275-ed584003e255 2026-02-20T18:02:55.785 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:55.784+0000 7f7eae00e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:55.787 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:55.786+0000 7f7eae00e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:55.788 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:55.787+0000 7f7eae00e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:55.788 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:55.787+0000 7f7eae00e900 -1 bdev(0x562a0fa30c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-20T18:02:55.788 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:55.787+0000 7f7eae00e900 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-20T18:02:56.111 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-20T18:02:56.112 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:02:56.112 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-20T18:02:56.112 INFO:tasks.workunit.client.0.trial195.stdout:adding osd1 key to auth repository 2026-02-20T18:02:56.113 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:02:56.437 INFO:tasks.workunit.client.0.trial195.stdout:start osd.1 2026-02-20T18:02:56.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-20T18:02:56.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:02:56.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:02:56.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:02:56.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:02:56.456 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:56.455+0000 7f03d7383900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:56.457 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:56.456+0000 7f03d7383900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:56.458 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:56.457+0000 7f03d7383900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:56.606 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:56.605+0000 7f03d7383900 -1 Falling back to public interface 2026-02-20T18:02:56.665 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-20T18:02:56.665 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:02:56.665 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-20T18:02:56.665 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:02:56.665 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:02:56.665 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:02:56.666 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:02:56.666 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:02:56.666 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:02:56.667 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:02:56.889 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:02:56.891 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:56.890+0000 7f03d7383900 -1 osd.1 0 log_to_monitors true 2026-02-20T18:02:57.891 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:02:57.891 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:02:57.892 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:02:57.893 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:02:57.893 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:02:57.893 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:02:58.121 INFO:tasks.workunit.client.0.trial195.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3796455933,v1:127.0.0.1:6811/3796455933] [v2:127.0.0.1:6812/3796455933,v1:127.0.0.1:6813/3796455933] exists,up a10f71dd-3057-48c1-8275-ed584003e255 2026-02-20T18:02:58.125 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:02:58.125 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:02:58.125 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:02:58.125 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:02:58.125 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-20T18:02:58.125 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:02:58.125 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:02:58.126 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-20T18:02:58.126 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:02:58.126 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-20T18:02:58.126 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-20T18:02:58.126 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:02:58.126 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:02:58.126 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:02:58.126 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-20T18:02:58.126 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-20T18:02:58.126 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:02:58.126 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:02:58.126 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:02:58.127 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:02:58.127 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:02:58.127 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:02:58.127 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:02:58.127 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:02:58.127 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:02:58.127 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:02:58.127 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:02:58.127 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:02:58.127 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:02:58.127 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:02:58.127 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:02:58.127 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:02:58.128 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:02:58.128 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:02:58.128 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:02:58.128 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:02:58.128 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:02:58.128 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:02:58.128 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-20T18:02:58.128 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:02:58.129 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a517993d-c5e6-4b8f-bae5-a9526233eff2 2026-02-20T18:02:58.129 INFO:tasks.workunit.client.0.trial195.stdout:add osd2 a517993d-c5e6-4b8f-bae5-a9526233eff2 2026-02-20T18:02:58.129 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 a517993d-c5e6-4b8f-bae5-a9526233eff2' 2026-02-20T18:02:58.129 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:02:58.138 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDSoZhpJ4E0CBAAjy8fifiEbzIFaKT4uMeTug== 2026-02-20T18:02:58.139 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDSoZhpJ4E0CBAAjy8fifiEbzIFaKT4uMeTug=="}' 2026-02-20T18:02:58.139 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a517993d-c5e6-4b8f-bae5-a9526233eff2 -i td/test-erasure-eio/2/new.json 2026-02-20T18:02:58.366 INFO:tasks.workunit.client.0.trial195.stdout:2 2026-02-20T18:02:58.374 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-20T18:02:58.375 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDSoZhpJ4E0CBAAjy8fifiEbzIFaKT4uMeTug== --osd-uuid a517993d-c5e6-4b8f-bae5-a9526233eff2 2026-02-20T18:02:58.395 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:58.394+0000 7f44f837d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:58.397 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:58.396+0000 7f44f837d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:58.398 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:58.397+0000 7f44f837d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:58.398 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:58.397+0000 7f44f837d900 -1 bdev(0x55ed63320c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-20T18:02:58.398 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:58.397+0000 7f44f837d900 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-20T18:02:58.722 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-20T18:02:58.722 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:02:58.723 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-20T18:02:58.723 INFO:tasks.workunit.client.0.trial195.stdout:adding osd2 key to auth repository 2026-02-20T18:02:58.723 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:02:59.041 INFO:tasks.workunit.client.0.trial195.stdout:start osd.2 2026-02-20T18:02:59.042 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-20T18:02:59.042 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:02:59.042 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:02:59.042 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:02:59.042 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:02:59.061 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:59.061+0000 7f8721e7f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:59.063 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:59.062+0000 7f8721e7f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:59.063 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:59.063+0000 7f8721e7f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:02:59.184 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:59.183+0000 7f8721e7f900 -1 Falling back to public interface 2026-02-20T18:02:59.267 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-20T18:02:59.267 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:02:59.267 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-20T18:02:59.267 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:02:59.267 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:02:59.267 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:02:59.267 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:02:59.267 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:02:59.267 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:02:59.267 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:02:59.495 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:02:59.531 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:02:59.530+0000 7f8721e7f900 -1 osd.2 0 log_to_monitors true 2026-02-20T18:03:00.496 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:03:00.497 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:03:00.497 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:03:00.498 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:03:00.498 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:03:00.498 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:03:00.726 INFO:tasks.workunit.client.0.trial195.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2311678804,v1:127.0.0.1:6819/2311678804] [v2:127.0.0.1:6820/2311678804,v1:127.0.0.1:6821/2311678804] exists,up a517993d-c5e6-4b8f-bae5-a9526233eff2 2026-02-20T18:03:00.730 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:03:00.731 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:03:00.731 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:03:00.731 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:03:00.731 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-20T18:03:00.731 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:03:00.731 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:03:00.731 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-20T18:03:00.731 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:03:00.731 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-20T18:03:00.731 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-20T18:03:00.732 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:03:00.732 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:03:00.732 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:03:00.732 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-20T18:03:00.732 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-20T18:03:00.732 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:03:00.732 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:03:00.732 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:03:00.732 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:03:00.732 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:03:00.732 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:03:00.732 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:03:00.733 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:03:00.733 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:03:00.733 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:03:00.733 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:03:00.733 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:03:00.733 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:03:00.733 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:03:00.733 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:03:00.733 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:03:00.733 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:03:00.733 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:03:00.733 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:03:00.734 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:03:00.734 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:03:00.734 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:03:00.734 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-20T18:03:00.734 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:03:00.734 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=cdea7076-00e5-499e-af29-de52282f956a 2026-02-20T18:03:00.734 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 cdea7076-00e5-499e-af29-de52282f956a' 2026-02-20T18:03:00.735 INFO:tasks.workunit.client.0.trial195.stdout:add osd3 cdea7076-00e5-499e-af29-de52282f956a 2026-02-20T18:03:00.736 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:03:00.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDUoZhpjo5GLBAAQKXaipMTodc2trAufAPqAA== 2026-02-20T18:03:00.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDUoZhpjo5GLBAAQKXaipMTodc2trAufAPqAA=="}' 2026-02-20T18:03:00.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new cdea7076-00e5-499e-af29-de52282f956a -i td/test-erasure-eio/3/new.json 2026-02-20T18:03:00.976 INFO:tasks.workunit.client.0.trial195.stdout:3 2026-02-20T18:03:00.984 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-20T18:03:00.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDUoZhpjo5GLBAAQKXaipMTodc2trAufAPqAA== --osd-uuid cdea7076-00e5-499e-af29-de52282f956a 2026-02-20T18:03:01.005 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:01.004+0000 7fbbf83cb900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:01.006 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:01.005+0000 7fbbf83cb900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:01.007 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:01.006+0000 7fbbf83cb900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:01.008 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:01.007+0000 7fbbf83cb900 -1 bdev(0x5560f9bacc00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-20T18:03:01.008 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:01.007+0000 7fbbf83cb900 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-20T18:03:01.326 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-20T18:03:01.326 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:03:01.327 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-20T18:03:01.327 INFO:tasks.workunit.client.0.trial195.stdout:adding osd3 key to auth repository 2026-02-20T18:03:01.328 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:03:01.651 INFO:tasks.workunit.client.0.trial195.stdout:start osd.3 2026-02-20T18:03:01.651 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-20T18:03:01.652 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:03:01.652 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:03:01.652 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:03:01.652 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:03:01.671 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:01.670+0000 7f2f97fdc900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:01.672 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:01.671+0000 7f2f97fdc900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:01.673 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:01.672+0000 7f2f97fdc900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:01.802 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:01.801+0000 7f2f97fdc900 -1 Falling back to public interface 2026-02-20T18:03:01.878 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-20T18:03:01.879 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:03:01.879 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-20T18:03:01.879 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:03:01.879 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:03:01.879 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:03:01.879 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:03:01.879 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:03:01.879 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:03:01.879 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T18:03:02.104 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:03:02.171 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:02.170+0000 7f2f97fdc900 -1 osd.3 0 log_to_monitors true 2026-02-20T18:03:03.105 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:03:03.106 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:03:03.106 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:03:03.106 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:03:03.106 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:03:03.106 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T18:03:03.332 INFO:tasks.workunit.client.0.trial195.stdout:osd.3 up in weight 1 up_from 25 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1588720816,v1:127.0.0.1:6827/1588720816] [v2:127.0.0.1:6828/1588720816,v1:127.0.0.1:6829/1588720816] exists,up cdea7076-00e5-499e-af29-de52282f956a 2026-02-20T18:03:03.336 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:03:03.336 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:03:03.336 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:03:03.336 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:03:03.336 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-20T18:03:03.336 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:03:03.336 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:03:03.336 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-20T18:03:03.336 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:03:03.337 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-20T18:03:03.337 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-20T18:03:03.337 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:03:03.337 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:03:03.337 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:03:03.337 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-20T18:03:03.337 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-20T18:03:03.337 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:03:03.337 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:03:03.337 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:03:03.337 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:03:03.338 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:03:03.338 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:03:03.338 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:03:03.338 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:03:03.338 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:03:03.338 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:03:03.338 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:03:03.338 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:03:03.338 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:03:03.338 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:03:03.338 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:03:03.338 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:03:03.338 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:03:03.339 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:03:03.339 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:03:03.339 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:03:03.339 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:03:03.339 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:03:03.339 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-20T18:03:03.339 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:03:03.339 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5c4a2815-26f5-4f35-8853-ebeb9af36c94 2026-02-20T18:03:03.339 INFO:tasks.workunit.client.0.trial195.stdout:add osd4 5c4a2815-26f5-4f35-8853-ebeb9af36c94 2026-02-20T18:03:03.340 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 5c4a2815-26f5-4f35-8853-ebeb9af36c94' 2026-02-20T18:03:03.340 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:03:03.349 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDXoZhpX43FFBAAKOjW22rQqjJKQZx56JnJ/A== 2026-02-20T18:03:03.349 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDXoZhpX43FFBAAKOjW22rQqjJKQZx56JnJ/A=="}' 2026-02-20T18:03:03.350 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5c4a2815-26f5-4f35-8853-ebeb9af36c94 -i td/test-erasure-eio/4/new.json 2026-02-20T18:03:03.582 INFO:tasks.workunit.client.0.trial195.stdout:4 2026-02-20T18:03:03.591 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-20T18:03:03.592 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDXoZhpX43FFBAAKOjW22rQqjJKQZx56JnJ/A== --osd-uuid 5c4a2815-26f5-4f35-8853-ebeb9af36c94 2026-02-20T18:03:03.612 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:03.612+0000 7fce83b1c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:03.614 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:03.613+0000 7fce83b1c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:03.615 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:03.614+0000 7fce83b1c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:03.615 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:03.614+0000 7fce83b1c900 -1 bdev(0x560bf01aec00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-20T18:03:03.615 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:03.615+0000 7fce83b1c900 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-20T18:03:03.956 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-20T18:03:03.957 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:03:03.958 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-20T18:03:03.958 INFO:tasks.workunit.client.0.trial195.stdout:adding osd4 key to auth repository 2026-02-20T18:03:03.959 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:03:04.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-20T18:03:04.278 INFO:tasks.workunit.client.0.trial195.stdout:start osd.4 2026-02-20T18:03:04.278 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:03:04.278 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:03:04.278 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:03:04.278 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:03:04.296 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:04.295+0000 7f02e3059900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:04.298 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:04.297+0000 7f02e3059900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:04.299 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:04.298+0000 7f02e3059900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:04.420 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:04.419+0000 7f02e3059900 -1 Falling back to public interface 2026-02-20T18:03:04.504 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-20T18:03:04.504 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:03:04.504 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-20T18:03:04.505 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:03:04.505 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:03:04.505 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:03:04.505 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:03:04.506 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:03:04.506 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:03:04.506 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-20T18:03:04.736 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:03:04.763 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:04.762+0000 7f02e3059900 -1 osd.4 0 log_to_monitors true 2026-02-20T18:03:05.737 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:03:05.737 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:03:05.737 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:03:05.737 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:03:05.738 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:03:05.738 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-20T18:03:05.968 INFO:tasks.workunit.client.0.trial195.stdout:osd.4 up in weight 1 up_from 34 up_thru 35 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/3203681582,v1:127.0.0.1:6835/3203681582] [v2:127.0.0.1:6836/3203681582,v1:127.0.0.1:6837/3203681582] exists,up 5c4a2815-26f5-4f35-8853-ebeb9af36c94 2026-02-20T18:03:05.972 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:03:05.972 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:03:05.972 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:03:05.973 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:03:05.973 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-20T18:03:05.973 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:03:05.973 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:03:05.973 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-20T18:03:05.973 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:03:05.973 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-20T18:03:05.973 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-20T18:03:05.973 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:03:05.973 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:03:05.973 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:03:05.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-20T18:03:05.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-20T18:03:05.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:03:05.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:03:05.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:03:05.974 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:03:05.974 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:03:05.974 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:03:05.974 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:03:05.975 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:03:05.975 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:03:05.975 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:03:05.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:03:05.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:03:05.976 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:03:05.976 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:03:05.976 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:03:05.976 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:03:05.976 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:03:05.976 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:03:05.976 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:03:05.976 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:03:05.977 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:03:05.977 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:03:05.977 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-20T18:03:05.977 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:03:05.977 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=79a1581f-eb95-4aa3-b2cf-5d802628458e 2026-02-20T18:03:05.977 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 79a1581f-eb95-4aa3-b2cf-5d802628458e' 2026-02-20T18:03:05.977 INFO:tasks.workunit.client.0.trial195.stdout:add osd5 79a1581f-eb95-4aa3-b2cf-5d802628458e 2026-02-20T18:03:05.978 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:03:05.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDZoZhpBKatOhAA3UbSCx4jzO9McsXEJXaJZA== 2026-02-20T18:03:05.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDZoZhpBKatOhAA3UbSCx4jzO9McsXEJXaJZA=="}' 2026-02-20T18:03:05.986 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 79a1581f-eb95-4aa3-b2cf-5d802628458e -i td/test-erasure-eio/5/new.json 2026-02-20T18:03:06.216 INFO:tasks.workunit.client.0.trial195.stdout:5 2026-02-20T18:03:06.223 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-20T18:03:06.224 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDZoZhpBKatOhAA3UbSCx4jzO9McsXEJXaJZA== --osd-uuid 79a1581f-eb95-4aa3-b2cf-5d802628458e 2026-02-20T18:03:06.245 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:06.244+0000 7fc2d0ed7900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:06.246 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:06.245+0000 7fc2d0ed7900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:06.247 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:06.246+0000 7fc2d0ed7900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:06.247 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:06.246+0000 7fc2d0ed7900 -1 bdev(0x55957586ec00 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-20T18:03:06.247 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:06.246+0000 7fc2d0ed7900 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-20T18:03:06.581 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-20T18:03:06.581 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:03:06.582 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-20T18:03:06.582 INFO:tasks.workunit.client.0.trial195.stdout:adding osd5 key to auth repository 2026-02-20T18:03:06.582 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:03:06.899 INFO:tasks.workunit.client.0.trial195.stdout:start osd.5 2026-02-20T18:03:06.900 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-20T18:03:06.900 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:03:06.901 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:03:06.901 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:03:06.901 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:03:06.919 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:06.918+0000 7f8cfb815900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:06.920 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:06.919+0000 7f8cfb815900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:06.921 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:06.920+0000 7f8cfb815900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:07.049 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:07.048+0000 7f8cfb815900 -1 Falling back to public interface 2026-02-20T18:03:07.128 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-20T18:03:07.128 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:03:07.128 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-20T18:03:07.128 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:03:07.128 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:03:07.129 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:03:07.129 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:03:07.130 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:03:07.130 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:03:07.130 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-20T18:03:07.351 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:07.350+0000 7f8cfb815900 -1 osd.5 0 log_to_monitors true 2026-02-20T18:03:07.357 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:03:08.359 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:03:08.359 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:03:08.359 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:03:08.360 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:03:08.360 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:03:08.360 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-20T18:03:08.584 INFO:tasks.workunit.client.0.trial195.stdout:osd.5 up in weight 1 up_from 43 up_thru 44 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/784397375,v1:127.0.0.1:6843/784397375] [v2:127.0.0.1:6844/784397375,v1:127.0.0.1:6845/784397375] exists,up 79a1581f-eb95-4aa3-b2cf-5d802628458e 2026-02-20T18:03:08.588 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:03:08.589 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:03:08.589 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:03:08.589 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:03:08.589 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-20T18:03:08.589 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:03:08.589 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:03:08.589 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-20T18:03:08.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:03:08.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-20T18:03:08.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-20T18:03:08.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:03:08.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:03:08.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:03:08.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-20T18:03:08.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-20T18:03:08.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:03:08.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:03:08.591 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:03:08.591 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:03:08.591 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:03:08.591 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:03:08.591 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:03:08.591 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:03:08.591 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:03:08.591 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:03:08.591 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:03:08.591 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:03:08.592 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:03:08.592 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:03:08.592 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:03:08.592 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:03:08.592 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:03:08.592 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:03:08.592 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:03:08.592 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:03:08.592 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:03:08.592 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:03:08.592 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-20T18:03:08.592 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:03:08.593 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8d073229-2de5-4414-9898-3f56d59ae544 2026-02-20T18:03:08.593 INFO:tasks.workunit.client.0.trial195.stdout:add osd6 8d073229-2de5-4414-9898-3f56d59ae544 2026-02-20T18:03:08.594 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 8d073229-2de5-4414-9898-3f56d59ae544' 2026-02-20T18:03:08.594 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:03:08.601 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDcoZhpdKPKIxAAfEOSmnzZja1EtJ+SjG7FpQ== 2026-02-20T18:03:08.602 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDcoZhpdKPKIxAAfEOSmnzZja1EtJ+SjG7FpQ=="}' 2026-02-20T18:03:08.602 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8d073229-2de5-4414-9898-3f56d59ae544 -i td/test-erasure-eio/6/new.json 2026-02-20T18:03:08.835 INFO:tasks.workunit.client.0.trial195.stdout:6 2026-02-20T18:03:08.842 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-20T18:03:08.843 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDcoZhpdKPKIxAAfEOSmnzZja1EtJ+SjG7FpQ== --osd-uuid 8d073229-2de5-4414-9898-3f56d59ae544 2026-02-20T18:03:08.863 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:08.861+0000 7f62f62fe900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:08.865 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:08.863+0000 7f62f62fe900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:08.866 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:08.864+0000 7f62f62fe900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:08.866 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:08.864+0000 7f62f62fe900 -1 bdev(0x55eb20e60c00 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-20T18:03:08.867 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:08.865+0000 7f62f62fe900 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-20T18:03:09.196 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-20T18:03:09.196 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:03:09.197 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-20T18:03:09.197 INFO:tasks.workunit.client.0.trial195.stdout:adding osd6 key to auth repository 2026-02-20T18:03:09.197 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:03:09.517 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-20T18:03:09.518 INFO:tasks.workunit.client.0.trial195.stdout:start osd.6 2026-02-20T18:03:09.519 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:03:09.519 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:03:09.519 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:03:09.519 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:03:09.537 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:09.535+0000 7f5ccdab8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:09.538 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:09.537+0000 7f5ccdab8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:09.540 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:09.538+0000 7f5ccdab8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:09.680 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:09.678+0000 7f5ccdab8900 -1 Falling back to public interface 2026-02-20T18:03:09.742 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-20T18:03:09.742 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:03:09.742 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-20T18:03:09.742 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:03:09.742 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:03:09.742 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:03:09.742 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:03:09.743 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:03:09.743 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:03:09.743 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-20T18:03:09.970 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:03:10.059 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:10.057+0000 7f5ccdab8900 -1 osd.6 0 log_to_monitors true 2026-02-20T18:03:10.971 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:03:10.972 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:03:10.972 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:03:10.973 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:03:10.973 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:03:10.973 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-20T18:03:11.198 INFO:tasks.workunit.client.0.trial195.stdout:osd.6 up in weight 1 up_from 49 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/3716909075,v1:127.0.0.1:6851/3716909075] [v2:127.0.0.1:6852/3716909075,v1:127.0.0.1:6853/3716909075] exists,up 8d073229-2de5-4414-9898-3f56d59ae544 2026-02-20T18:03:11.199 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:03:11.199 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:03:11.199 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:03:11.200 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-20T18:03:11.200 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-20T18:03:11.200 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-20T18:03:11.200 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:03:11.200 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:03:11.200 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:03:11.200 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.0.asok 2026-02-20T18:03:11.200 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-20T18:03:11.200 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-osd.0.asok log flush 2026-02-20T18:03:11.247 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T18:03:11.251 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-20T18:03:11.254 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T18:02:54.295+0000 7f9a665da900 0 load: jerasure load: lrc 2026-02-20T18:03:11.255 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:477: TEST_ec_recovery_multiple_objects: CEPH_ARGS='--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:03:11.255 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:479: TEST_ec_recovery_multiple_objects: local poolname=pool-jerasure 2026-02-20T18:03:11.256 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:480: TEST_ec_recovery_multiple_objects: create_erasure_coded_pool pool-jerasure 3 2 2026-02-20T18:03:11.256 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-20T18:03:11.256 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-20T18:03:11.256 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-20T18:03:11.256 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-20T18:03:11.256 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-20T18:03:11.256 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-20T18:03:11.256 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-20T18:03:11.539 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-20T18:03:11.539 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-20T18:03:11.915 INFO:tasks.workunit.client.0.trial195.stderr:pool 'pool-jerasure' already exists 2026-02-20T18:03:11.923 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-20T18:03:12.925 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-20T18:03:12.925 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:03:12.925 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:03:12.925 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:03:12.925 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:03:12.926 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:03:12.926 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:03:12.926 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:03:12.926 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:03:12.926 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:03:13.011 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:03:13.012 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:03:13.012 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:03:13.012 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:03:13.012 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:03:13.012 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:03:13.235 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:03:13.235 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:03:13.235 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:03:13.235 INFO:tasks.workunit.client.0.trial195.stderr:3 2026-02-20T18:03:13.235 INFO:tasks.workunit.client.0.trial195.stderr:4 2026-02-20T18:03:13.235 INFO:tasks.workunit.client.0.trial195.stderr:5 2026-02-20T18:03:13.235 INFO:tasks.workunit.client.0.trial195.stderr:6' 2026-02-20T18:03:13.236 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:03:13.236 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:03:13.236 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:03:13.299 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803783 2026-02-20T18:03:13.299 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803783 2026-02-20T18:03:13.299 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803783' 2026-02-20T18:03:13.300 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:03:13.300 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:03:13.365 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574854 2026-02-20T18:03:13.365 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574854 2026-02-20T18:03:13.365 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803783 1-55834574854' 2026-02-20T18:03:13.366 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:03:13.366 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:03:13.429 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411333 2026-02-20T18:03:13.429 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411333 2026-02-20T18:03:13.429 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803783 1-55834574854 2-77309411333' 2026-02-20T18:03:13.429 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:03:13.429 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:03:13.495 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182405 2026-02-20T18:03:13.495 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182405 2026-02-20T18:03:13.495 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803783 1-55834574854 2-77309411333 3-107374182405' 2026-02-20T18:03:13.495 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:03:13.495 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-20T18:03:13.560 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888068 2026-02-20T18:03:13.560 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888068 2026-02-20T18:03:13.560 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803783 1-55834574854 2-77309411333 3-107374182405 4-146028888068' 2026-02-20T18:03:13.561 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:03:13.561 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-20T18:03:13.625 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593732 2026-02-20T18:03:13.625 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593732 2026-02-20T18:03:13.625 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803783 1-55834574854 2-77309411333 3-107374182405 4-146028888068 5-184683593732' 2026-02-20T18:03:13.625 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:03:13.625 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-20T18:03:13.693 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=210453397507 2026-02-20T18:03:13.693 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 210453397507 2026-02-20T18:03:13.693 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803783 1-55834574854 2-77309411333 3-107374182405 4-146028888068 5-184683593732 6-210453397507' 2026-02-20T18:03:13.693 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:03:13.694 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803783 2026-02-20T18:03:13.694 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:03:13.694 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:03:13.695 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803783 2026-02-20T18:03:13.695 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:03:13.696 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803783 2026-02-20T18:03:13.696 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803783' 2026-02-20T18:03:13.696 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.0 seq 25769803783 2026-02-20T18:03:13.697 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:03:13.918 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803781 -lt 25769803783 2026-02-20T18:03:13.918 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:03:14.919 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:03:14.919 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:03:15.143 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803783 -lt 25769803783 2026-02-20T18:03:15.143 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:03:15.143 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574854 2026-02-20T18:03:15.143 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:03:15.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:03:15.145 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-55834574854 2026-02-20T18:03:15.146 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:03:15.146 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574854 2026-02-20T18:03:15.147 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.1 seq 55834574854 2026-02-20T18:03:15.147 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 55834574854' 2026-02-20T18:03:15.147 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:03:15.374 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574854 -lt 55834574854 2026-02-20T18:03:15.374 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:03:15.375 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411333 2026-02-20T18:03:15.375 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:03:15.376 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:03:15.376 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-77309411333 2026-02-20T18:03:15.376 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:03:15.377 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=77309411333 2026-02-20T18:03:15.377 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 77309411333' 2026-02-20T18:03:15.378 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.2 seq 77309411333 2026-02-20T18:03:15.378 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:03:15.604 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411333 -lt 77309411333 2026-02-20T18:03:15.604 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:03:15.604 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182405 2026-02-20T18:03:15.604 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:03:15.605 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:03:15.606 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-107374182405 2026-02-20T18:03:15.606 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:03:15.607 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=107374182405 2026-02-20T18:03:15.607 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 107374182405' 2026-02-20T18:03:15.607 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.3 seq 107374182405 2026-02-20T18:03:15.608 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:03:15.830 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 107374182405 -lt 107374182405 2026-02-20T18:03:15.830 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:03:15.831 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-146028888068 2026-02-20T18:03:15.831 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:03:15.832 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-20T18:03:15.832 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-146028888068 2026-02-20T18:03:15.832 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:03:15.833 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888068 2026-02-20T18:03:15.834 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.4 seq 146028888068 2026-02-20T18:03:15.834 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 146028888068' 2026-02-20T18:03:15.834 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-20T18:03:16.060 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888068 -lt 146028888068 2026-02-20T18:03:16.060 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:03:16.061 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-184683593732 2026-02-20T18:03:16.061 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:03:16.061 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-20T18:03:16.062 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-184683593732 2026-02-20T18:03:16.062 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:03:16.063 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=184683593732 2026-02-20T18:03:16.063 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 184683593732' 2026-02-20T18:03:16.064 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.5 seq 184683593732 2026-02-20T18:03:16.064 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-20T18:03:16.289 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 184683593732 -lt 184683593732 2026-02-20T18:03:16.289 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:03:16.290 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-210453397507 2026-02-20T18:03:16.290 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:03:16.290 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-20T18:03:16.291 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-210453397507 2026-02-20T18:03:16.291 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:03:16.292 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=210453397507 2026-02-20T18:03:16.292 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 210453397507' 2026-02-20T18:03:16.293 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.6 seq 210453397507 2026-02-20T18:03:16.293 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-20T18:03:16.516 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 210453397507 -lt 210453397507 2026-02-20T18:03:16.517 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:03:16.517 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:03:16.517 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:03:16.829 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:03:16.830 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:03:16.830 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:03:16.830 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:03:16.830 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:03:16.830 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:03:16.830 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:03:16.830 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:03:17.048 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:03:17.048 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:03:17.048 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:03:17.049 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:03:17.363 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:03:17.363 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:03:17.363 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:03:17.363 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:482: TEST_ec_recovery_multiple_objects: rados_put td/test-erasure-eio pool-jerasure test1 2026-02-20T18:03:17.364 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-20T18:03:17.364 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-20T18:03:17.364 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test1 2026-02-20T18:03:17.364 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:03:17.364 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-20T18:03:17.364 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:03:17.364 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-20T18:03:17.364 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:03:17.364 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-20T18:03:17.364 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:03:17.364 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-20T18:03:17.364 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test1 td/test-erasure-eio/ORIGINAL 2026-02-20T18:03:17.383 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:483: TEST_ec_recovery_multiple_objects: rados_put td/test-erasure-eio pool-jerasure test2 2026-02-20T18:03:17.383 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-20T18:03:17.384 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-20T18:03:17.384 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test2 2026-02-20T18:03:17.384 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:03:17.384 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-20T18:03:17.384 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:03:17.384 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-20T18:03:17.384 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:03:17.384 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-20T18:03:17.384 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:03:17.384 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-20T18:03:17.384 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test2 td/test-erasure-eio/ORIGINAL 2026-02-20T18:03:17.404 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:484: TEST_ec_recovery_multiple_objects: rados_put td/test-erasure-eio pool-jerasure test3 2026-02-20T18:03:17.404 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-20T18:03:17.404 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-20T18:03:17.404 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test3 2026-02-20T18:03:17.404 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:03:17.404 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-20T18:03:17.404 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:03:17.404 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-20T18:03:17.405 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:03:17.405 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-20T18:03:17.405 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:03:17.405 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-20T18:03:17.405 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test3 td/test-erasure-eio/ORIGINAL 2026-02-20T18:03:17.423 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:486: TEST_ec_recovery_multiple_objects: ceph osd out 0 2026-02-20T18:03:17.702 INFO:tasks.workunit.client.0.trial195.stderr:osd.0 is already out. 2026-02-20T18:03:17.711 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:489: TEST_ec_recovery_multiple_objects: wait_for_clean 2026-02-20T18:03:17.711 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:03:17.711 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:03:17.711 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:03:17.711 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:03:17.712 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:03:17.712 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:03:17.712 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:03:17.712 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:03:17.712 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:03:17.797 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:03:17.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:03:17.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:03:17.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:03:17.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:03:17.798 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:03:18.024 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:03:18.024 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:03:18.025 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:03:18.025 INFO:tasks.workunit.client.0.trial195.stderr:3 2026-02-20T18:03:18.025 INFO:tasks.workunit.client.0.trial195.stderr:4 2026-02-20T18:03:18.025 INFO:tasks.workunit.client.0.trial195.stderr:5 2026-02-20T18:03:18.025 INFO:tasks.workunit.client.0.trial195.stderr:6' 2026-02-20T18:03:18.025 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:03:18.025 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:03:18.025 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:03:18.092 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803786 2026-02-20T18:03:18.092 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803786 2026-02-20T18:03:18.092 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786' 2026-02-20T18:03:18.092 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:03:18.092 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:03:18.157 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574857 2026-02-20T18:03:18.157 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574857 2026-02-20T18:03:18.157 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786 1-55834574857' 2026-02-20T18:03:18.157 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:03:18.157 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:03:18.222 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411336 2026-02-20T18:03:18.222 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411336 2026-02-20T18:03:18.222 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786 1-55834574857 2-77309411336' 2026-02-20T18:03:18.222 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:03:18.222 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:03:18.288 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182407 2026-02-20T18:03:18.288 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182407 2026-02-20T18:03:18.288 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786 1-55834574857 2-77309411336 3-107374182407' 2026-02-20T18:03:18.288 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:03:18.288 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-20T18:03:18.355 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888071 2026-02-20T18:03:18.356 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888071 2026-02-20T18:03:18.356 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786 1-55834574857 2-77309411336 3-107374182407 4-146028888071' 2026-02-20T18:03:18.356 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:03:18.356 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-20T18:03:18.422 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593734 2026-02-20T18:03:18.422 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593734 2026-02-20T18:03:18.422 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786 1-55834574857 2-77309411336 3-107374182407 4-146028888071 5-184683593734' 2026-02-20T18:03:18.422 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:03:18.422 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-20T18:03:18.487 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=210453397510 2026-02-20T18:03:18.487 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 210453397510 2026-02-20T18:03:18.487 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786 1-55834574857 2-77309411336 3-107374182407 4-146028888071 5-184683593734 6-210453397510' 2026-02-20T18:03:18.487 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:03:18.487 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803786 2026-02-20T18:03:18.487 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:03:18.488 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:03:18.488 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803786 2026-02-20T18:03:18.488 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:03:18.489 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803786 2026-02-20T18:03:18.489 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803786' 2026-02-20T18:03:18.490 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.0 seq 25769803786 2026-02-20T18:03:18.490 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:03:18.713 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803786 -lt 25769803786 2026-02-20T18:03:18.714 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:03:18.714 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574857 2026-02-20T18:03:18.714 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:03:18.715 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:03:18.716 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-55834574857 2026-02-20T18:03:18.716 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:03:18.717 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574857 2026-02-20T18:03:18.717 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 55834574857' 2026-02-20T18:03:18.717 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.1 seq 55834574857 2026-02-20T18:03:18.718 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:03:18.942 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574857 -lt 55834574857 2026-02-20T18:03:18.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:03:18.943 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411336 2026-02-20T18:03:18.943 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:03:18.944 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:03:18.944 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-77309411336 2026-02-20T18:03:18.945 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:03:18.946 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=77309411336 2026-02-20T18:03:18.946 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 77309411336' 2026-02-20T18:03:18.946 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.2 seq 77309411336 2026-02-20T18:03:18.946 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:03:19.174 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411334 -lt 77309411336 2026-02-20T18:03:19.174 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:03:20.175 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:03:20.175 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:03:20.401 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411336 -lt 77309411336 2026-02-20T18:03:20.401 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:03:20.401 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182407 2026-02-20T18:03:20.401 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:03:20.402 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:03:20.403 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-107374182407 2026-02-20T18:03:20.403 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:03:20.404 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=107374182407 2026-02-20T18:03:20.404 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 107374182407' 2026-02-20T18:03:20.404 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.3 seq 107374182407 2026-02-20T18:03:20.405 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:03:20.631 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 107374182408 -lt 107374182407 2026-02-20T18:03:20.631 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:03:20.632 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-146028888071 2026-02-20T18:03:20.632 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:03:20.633 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-20T18:03:20.633 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-146028888071 2026-02-20T18:03:20.633 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:03:20.635 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888071 2026-02-20T18:03:20.635 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 146028888071' 2026-02-20T18:03:20.635 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.4 seq 146028888071 2026-02-20T18:03:20.635 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-20T18:03:20.860 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888071 -lt 146028888071 2026-02-20T18:03:20.861 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:03:20.861 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-184683593734 2026-02-20T18:03:20.861 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:03:20.862 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-20T18:03:20.862 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-184683593734 2026-02-20T18:03:20.863 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:03:20.863 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=184683593734 2026-02-20T18:03:20.863 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 184683593734' 2026-02-20T18:03:20.864 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.5 seq 184683593734 2026-02-20T18:03:20.864 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-20T18:03:21.090 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 184683593735 -lt 184683593734 2026-02-20T18:03:21.090 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:03:21.091 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-210453397510 2026-02-20T18:03:21.091 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:03:21.091 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-20T18:03:21.092 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-210453397510 2026-02-20T18:03:21.092 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:03:21.093 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=210453397510 2026-02-20T18:03:21.093 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 210453397510' 2026-02-20T18:03:21.093 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.6 seq 210453397510 2026-02-20T18:03:21.094 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-20T18:03:21.317 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 210453397510 -lt 210453397510 2026-02-20T18:03:21.317 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:03:21.318 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:03:21.318 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:03:21.629 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:03:21.629 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:03:21.629 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:03:21.629 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:03:21.629 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:03:21.630 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:03:21.630 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:03:21.630 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:03:21.850 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-20T18:03:21.851 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:03:21.851 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:03:21.851 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:03:22.163 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-20T18:03:22.163 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' -1 2026-02-20T18:03:22.163 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-20T18:03:22.163 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=4 2026-02-20T18:03:22.163 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-20T18:03:22.163 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-20T18:03:22.264 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-20T18:03:22.264 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:03:22.265 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:03:22.265 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:03:22.265 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:03:22.265 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:03:22.265 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:03:22.265 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:03:22.485 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-20T18:03:22.485 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:03:22.485 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:03:22.486 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:03:22.797 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-20T18:03:22.797 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' 4 2026-02-20T18:03:22.797 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-20T18:03:22.797 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-20T18:03:22.797 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-20T18:03:22.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-20T18:03:22.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-20T18:03:22.798 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-20T18:03:22.798 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-20T18:03:23.108 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-20T18:03:23.108 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-20T18:03:23.108 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 13 )) 2026-02-20T18:03:23.108 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-20T18:03:23.108 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-20T18:03:23.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-20T18:03:23.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:03:23.309 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:03:23.309 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:03:23.309 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:03:23.309 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:03:23.309 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:03:23.310 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:03:23.534 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-20T18:03:23.534 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:03:23.535 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:03:23.535 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:03:23.847 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-20T18:03:23.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' 4 2026-02-20T18:03:23.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-20T18:03:23.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-20T18:03:23.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-20T18:03:23.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-20T18:03:23.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-20T18:03:23.848 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-20T18:03:23.848 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-20T18:03:24.162 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-20T18:03:24.162 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-20T18:03:24.163 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 2 >= 13 )) 2026-02-20T18:03:24.163 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-20T18:03:24.163 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.4 2026-02-20T18:03:24.562 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-20T18:03:24.563 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:03:24.564 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:03:24.564 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:03:24.564 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:03:24.564 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:03:24.564 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:03:24.564 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:03:24.782 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:03:24.782 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:03:24.783 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:03:24.783 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:03:25.093 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:03:25.093 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:03:25.093 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:03:25.093 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:491: TEST_ec_recovery_multiple_objects: rados_get td/test-erasure-eio pool-jerasure test1 2026-02-20T18:03:25.093 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-20T18:03:25.094 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-20T18:03:25.094 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test1 2026-02-20T18:03:25.094 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-20T18:03:25.094 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-20T18:03:25.094 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test1 td/test-erasure-eio/COPY 2026-02-20T18:03:25.112 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-20T18:03:25.112 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-20T18:03:25.114 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:492: TEST_ec_recovery_multiple_objects: rados_get td/test-erasure-eio pool-jerasure test2 2026-02-20T18:03:25.114 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-20T18:03:25.114 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-20T18:03:25.114 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test2 2026-02-20T18:03:25.114 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-20T18:03:25.114 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-20T18:03:25.114 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test2 td/test-erasure-eio/COPY 2026-02-20T18:03:25.134 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-20T18:03:25.134 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-20T18:03:25.136 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:493: TEST_ec_recovery_multiple_objects: rados_get td/test-erasure-eio pool-jerasure test3 2026-02-20T18:03:25.136 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-20T18:03:25.136 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-20T18:03:25.136 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test3 2026-02-20T18:03:25.136 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-20T18:03:25.136 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-20T18:03:25.136 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test3 td/test-erasure-eio/COPY 2026-02-20T18:03:25.154 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-20T18:03:25.155 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-20T18:03:25.155 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:495: TEST_ec_recovery_multiple_objects: delete_erasure_coded_pool pool-jerasure 2026-02-20T18:03:25.155 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-20T18:03:25.155 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-20T18:03:25.458 INFO:tasks.workunit.client.0.trial195.stderr:pool 'pool-jerasure' does not exist 2026-02-20T18:03:25.467 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-20T18:03:25.745 INFO:tasks.workunit.client.0.trial195.stderr:erasure-code-profile myprofile does not exist 2026-02-20T18:03:25.754 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-20T18:03:25.754 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-20T18:03:25.754 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T18:03:25.754 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-20T18:03:25.754 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:03:25.755 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:03:25.755 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:03:25.755 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:03:25.755 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:03:25.865 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:03:25.866 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T18:03:25.866 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T18:03:25.866 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T18:03:25.867 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T18:03:25.868 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T18:03:25.868 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T18:03:25.869 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:03:25.869 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T18:03:25.869 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T18:03:25.869 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:03:25.870 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T18:03:25.871 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T18:03:25.871 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-20T18:03:25.885 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T18:03:25.885 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:03:25.885 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:03:25.885 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.85808 2026-02-20T18:03:25.887 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T18:03:25.887 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T18:03:25.887 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-20T18:03:25.887 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-20T18:03:25.887 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-20T18:03:25.888 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-20T18:03:25.888 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-20T18:03:25.888 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T18:03:25.888 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-20T18:03:25.888 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:03:25.888 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:03:25.888 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:03:25.888 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:03:25.888 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:03:25.889 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:03:25.889 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T18:03:25.889 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T18:03:25.890 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T18:03:25.890 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T18:03:25.891 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T18:03:25.891 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T18:03:25.892 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:03:25.892 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T18:03:25.892 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T18:03:25.892 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:03:25.893 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T18:03:25.894 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T18:03:25.894 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-20T18:03:25.895 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T18:03:25.895 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:03:25.895 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:03:25.895 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.85808 2026-02-20T18:03:25.896 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T18:03:25.896 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T18:03:25.896 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-20T18:03:25.897 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-20T18:03:25.898 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:03:25.898 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:03:25.898 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.85808 2026-02-20T18:03:25.899 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-20T18:03:25.900 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-20T18:03:25.900 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-20T18:03:25.900 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-20T18:03:25.900 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-20T18:03:25.900 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-20T18:03:25.900 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-20T18:03:25.900 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-20T18:03:25.901 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-20T18:03:25.901 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-20T18:03:25.901 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-20T18:03:25.921 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-20T18:03:25.921 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:03:25.921 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:03:25.921 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:03:25.921 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:03:25.921 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:03:25.921 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:03:25.922 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-20T18:03:25.948 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-20T18:03:25.949 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-20T18:03:25.949 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T18:03:25.949 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T18:03:25.949 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-20T18:03:25.949 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-20T18:03:25.950 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T18:03:25.950 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:03:25.950 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:03:25.951 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:03:25.951 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:03:25.951 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:03:25.951 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:03:25.951 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T18:03:25.951 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.85808/ceph-mon.a.asok config get fsid 2026-02-20T18:03:25.994 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-20T18:03:25.994 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T18:03:25.994 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T18:03:25.994 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-20T18:03:25.995 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-20T18:03:25.995 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T18:03:25.995 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:03:25.995 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:03:25.995 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:03:25.995 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:03:25.995 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:03:25.996 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:03:25.996 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T18:03:25.996 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.85808/ceph-mon.a.asok config get mon_host 2026-02-20T18:03:26.041 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-20T18:03:26.041 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-20T18:03:26.041 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-20T18:03:26.041 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-20T18:03:26.041 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-20T18:03:26.041 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-20T18:03:26.042 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-20T18:03:26.140 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-20T18:03:26.141 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:03:26.141 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:03:26.141 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:03:26.141 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:03:26.141 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:03:26.141 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:03:26.142 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T18:03:26.142 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T18:03:26.159 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-20T18:03:26.159 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-20T18:03:26.257 INFO:tasks.workunit.client.0.trial195.stderr:pool 'rbd' already exists 2026-02-20T18:03:26.263 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-20T18:03:27.265 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-20T18:03:27.266 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:03:27.266 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:03:27.266 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:03:27.266 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:03:27.266 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:03:27.266 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:03:27.267 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-20T18:03:27.267 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-mon.a.asok log flush 2026-02-20T18:03:27.309 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T18:03:27.314 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-20T18:03:27.315 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T18:03:25.938+0000 7f1af9c09d80 0 load: jerasure load: lrc 2026-02-20T18:03:27.316 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_recovery_multiple_objects_eio td/test-erasure-eio 2026-02-20T18:03:27.317 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:500: TEST_ec_recovery_multiple_objects_eio: local dir=td/test-erasure-eio 2026-02-20T18:03:27.317 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:501: TEST_ec_recovery_multiple_objects_eio: local objname=myobject 2026-02-20T18:03:27.317 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:503: TEST_ec_recovery_multiple_objects_eio: ORIG_ARGS='--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:03:27.317 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:504: TEST_ec_recovery_multiple_objects_eio: CEPH_ARGS+=' --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-20T18:03:27.317 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:505: TEST_ec_recovery_multiple_objects_eio: setup_osds 7 2026-02-20T18:03:27.317 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=7 2026-02-20T18:03:27.317 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-20T18:03:27.317 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 7 - 1 2026-02-20T18:03:27.321 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 6 2026-02-20T18:03:27.321 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:03:27.321 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-20T18:03:27.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:03:27.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:03:27.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-20T18:03:27.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:03:27.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-20T18:03:27.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-20T18:03:27.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:03:27.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:03:27.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:03:27.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-20T18:03:27.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-20T18:03:27.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:03:27.323 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:03:27.323 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:03:27.323 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:03:27.323 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:03:27.323 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:03:27.323 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:03:27.323 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:03:27.323 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:03:27.323 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:03:27.323 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:03:27.323 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:03:27.323 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:03:27.324 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:03:27.324 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:03:27.324 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:03:27.324 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:03:27.324 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:03:27.324 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:03:27.324 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:03:27.324 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:03:27.324 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:03:27.324 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-20T18:03:27.324 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:03:27.325 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=73ba6b51-040b-455d-9993-eb8caa48a6ee 2026-02-20T18:03:27.325 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 73ba6b51-040b-455d-9993-eb8caa48a6ee' 2026-02-20T18:03:27.325 INFO:tasks.workunit.client.0.trial195.stdout:add osd0 73ba6b51-040b-455d-9993-eb8caa48a6ee 2026-02-20T18:03:27.326 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:03:27.335 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDvoZhpR2jqExAA1yOdn21OauZqteTuS6l5HA== 2026-02-20T18:03:27.335 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDvoZhpR2jqExAA1yOdn21OauZqteTuS6l5HA=="}' 2026-02-20T18:03:27.335 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 73ba6b51-040b-455d-9993-eb8caa48a6ee -i td/test-erasure-eio/0/new.json 2026-02-20T18:03:27.443 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:03:27.450 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-20T18:03:27.451 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDvoZhpR2jqExAA1yOdn21OauZqteTuS6l5HA== --osd-uuid 73ba6b51-040b-455d-9993-eb8caa48a6ee 2026-02-20T18:03:27.471 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:27.469+0000 7f4fac40c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:27.472 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:27.471+0000 7f4fac40c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:27.473 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:27.472+0000 7f4fac40c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:27.474 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:27.472+0000 7f4fac40c900 -1 bdev(0x561c6a5ccc00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-20T18:03:27.474 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:27.472+0000 7f4fac40c900 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-20T18:03:27.901 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-20T18:03:27.901 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:03:27.901 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-20T18:03:27.901 INFO:tasks.workunit.client.0.trial195.stdout:adding osd0 key to auth repository 2026-02-20T18:03:27.902 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:03:28.007 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-20T18:03:28.007 INFO:tasks.workunit.client.0.trial195.stdout:start osd.0 2026-02-20T18:03:28.007 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:03:28.007 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:03:28.007 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:03:28.007 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:03:28.026 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:28.025+0000 7f18f92ad900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:28.028 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:28.026+0000 7f18f92ad900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:28.029 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:28.027+0000 7f18f92ad900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:28.108 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-20T18:03:28.108 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:03:28.108 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-20T18:03:28.108 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:03:28.108 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:03:28.108 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:03:28.108 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:03:28.109 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:03:28.109 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:03:28.109 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T18:03:28.210 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:03:28.224 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:28.222+0000 7f18f92ad900 -1 Falling back to public interface 2026-02-20T18:03:28.497 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:28.495+0000 7f18f92ad900 -1 osd.0 0 log_to_monitors true 2026-02-20T18:03:29.211 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:03:29.211 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:03:29.212 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:03:29.212 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:03:29.212 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:03:29.213 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T18:03:29.441 INFO:tasks.workunit.client.0.trial195.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1702639787,v1:127.0.0.1:6803/1702639787] [v2:127.0.0.1:6804/1702639787,v1:127.0.0.1:6805/1702639787] exists,up 73ba6b51-040b-455d-9993-eb8caa48a6ee 2026-02-20T18:03:29.448 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:03:29.448 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:03:29.449 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:03:29.449 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:03:29.449 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-20T18:03:29.449 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:03:29.449 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:03:29.449 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-20T18:03:29.449 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:03:29.449 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-20T18:03:29.450 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-20T18:03:29.450 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:03:29.450 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:03:29.450 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:03:29.450 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-20T18:03:29.450 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-20T18:03:29.450 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:03:29.450 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:03:29.451 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:03:29.451 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:03:29.451 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:03:29.451 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:03:29.451 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:03:29.451 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:03:29.451 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:03:29.451 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:03:29.451 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:03:29.451 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:03:29.451 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:03:29.451 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:03:29.452 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:03:29.452 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:03:29.452 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:03:29.452 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:03:29.452 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:03:29.452 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:03:29.453 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:03:29.453 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:03:29.453 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-20T18:03:29.453 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:03:29.453 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2b1b966c-6bd4-4974-a527-e9c6d97c3c9e 2026-02-20T18:03:29.453 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 2b1b966c-6bd4-4974-a527-e9c6d97c3c9e' 2026-02-20T18:03:29.454 INFO:tasks.workunit.client.0.trial195.stdout:add osd1 2b1b966c-6bd4-4974-a527-e9c6d97c3c9e 2026-02-20T18:03:29.454 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:03:29.459 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDxoZhpUQtHGxAAwJ4/q56MHl7PIlgajD46nQ== 2026-02-20T18:03:29.459 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDxoZhpUQtHGxAAwJ4/q56MHl7PIlgajD46nQ=="}' 2026-02-20T18:03:29.459 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2b1b966c-6bd4-4974-a527-e9c6d97c3c9e -i td/test-erasure-eio/1/new.json 2026-02-20T18:03:29.690 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:03:29.697 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-20T18:03:29.698 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDxoZhpUQtHGxAAwJ4/q56MHl7PIlgajD46nQ== --osd-uuid 2b1b966c-6bd4-4974-a527-e9c6d97c3c9e 2026-02-20T18:03:29.719 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:29.717+0000 7f48ed014900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:29.720 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:29.719+0000 7f48ed014900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:29.721 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:29.720+0000 7f48ed014900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:29.722 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:29.720+0000 7f48ed014900 -1 bdev(0x55826d72cc00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-20T18:03:29.722 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:29.720+0000 7f48ed014900 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-20T18:03:30.032 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-20T18:03:30.033 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:03:30.033 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-20T18:03:30.034 INFO:tasks.workunit.client.0.trial195.stdout:adding osd1 key to auth repository 2026-02-20T18:03:30.034 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:03:30.358 INFO:tasks.workunit.client.0.trial195.stdout:start osd.1 2026-02-20T18:03:30.359 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-20T18:03:30.359 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:03:30.359 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:03:30.359 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:03:30.359 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:03:30.377 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:30.376+0000 7fa163e14900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:30.378 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:30.376+0000 7fa163e14900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:30.379 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:30.377+0000 7fa163e14900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:30.517 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:30.515+0000 7fa163e14900 -1 Falling back to public interface 2026-02-20T18:03:30.586 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-20T18:03:30.586 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:03:30.586 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-20T18:03:30.587 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:03:30.587 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:03:30.587 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:03:30.588 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:03:30.588 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:03:30.588 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:03:30.588 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:03:30.794 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:30.793+0000 7fa163e14900 -1 osd.1 0 log_to_monitors true 2026-02-20T18:03:30.818 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:03:31.819 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:03:31.820 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:03:31.820 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:03:31.821 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:03:31.821 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:03:31.821 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:03:32.048 INFO:tasks.workunit.client.0.trial195.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2024649108,v1:127.0.0.1:6811/2024649108] [v2:127.0.0.1:6812/2024649108,v1:127.0.0.1:6813/2024649108] exists,up 2b1b966c-6bd4-4974-a527-e9c6d97c3c9e 2026-02-20T18:03:32.049 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:03:32.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:03:32.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:03:32.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:03:32.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-20T18:03:32.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:03:32.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:03:32.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-20T18:03:32.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:03:32.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-20T18:03:32.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-20T18:03:32.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:03:32.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:03:32.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:03:32.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-20T18:03:32.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-20T18:03:32.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:03:32.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:03:32.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:03:32.050 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:03:32.050 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:03:32.050 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:03:32.050 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:03:32.050 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:03:32.050 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:03:32.051 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:03:32.051 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:03:32.051 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:03:32.051 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:03:32.051 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:03:32.051 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:03:32.051 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:03:32.051 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:03:32.051 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:03:32.051 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:03:32.051 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:03:32.051 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:03:32.051 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:03:32.051 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-20T18:03:32.052 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:03:32.052 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=fb1bbc15-0a94-45be-befa-f2bab0935b32 2026-02-20T18:03:32.052 INFO:tasks.workunit.client.0.trial195.stdout:add osd2 fb1bbc15-0a94-45be-befa-f2bab0935b32 2026-02-20T18:03:32.052 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 fb1bbc15-0a94-45be-befa-f2bab0935b32' 2026-02-20T18:03:32.052 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:03:32.066 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD0oZhpjl/fAxAAdY0lU7crx1n1lN+vm+aBgw== 2026-02-20T18:03:32.066 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD0oZhpjl/fAxAAdY0lU7crx1n1lN+vm+aBgw=="}' 2026-02-20T18:03:32.066 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new fb1bbc15-0a94-45be-befa-f2bab0935b32 -i td/test-erasure-eio/2/new.json 2026-02-20T18:03:32.296 INFO:tasks.workunit.client.0.trial195.stdout:2 2026-02-20T18:03:32.303 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-20T18:03:32.304 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD0oZhpjl/fAxAAdY0lU7crx1n1lN+vm+aBgw== --osd-uuid fb1bbc15-0a94-45be-befa-f2bab0935b32 2026-02-20T18:03:32.325 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:32.324+0000 7f234766b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:32.326 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:32.325+0000 7f234766b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:32.327 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:32.326+0000 7f234766b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:32.328 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:32.327+0000 7f234766b900 -1 bdev(0x562b2258ec00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-20T18:03:32.328 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:32.327+0000 7f234766b900 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-20T18:03:32.647 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-20T18:03:32.647 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:03:32.648 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-20T18:03:32.648 INFO:tasks.workunit.client.0.trial195.stdout:adding osd2 key to auth repository 2026-02-20T18:03:32.649 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:03:32.966 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-20T18:03:32.967 INFO:tasks.workunit.client.0.trial195.stdout:start osd.2 2026-02-20T18:03:32.968 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:03:32.968 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:03:32.968 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:03:32.968 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:03:32.985 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:32.984+0000 7f49bec23900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:32.987 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:32.986+0000 7f49bec23900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:32.988 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:32.987+0000 7f49bec23900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:33.123 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:33.121+0000 7f49bec23900 -1 Falling back to public interface 2026-02-20T18:03:33.195 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-20T18:03:33.195 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:03:33.196 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-20T18:03:33.196 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:03:33.196 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:03:33.196 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:03:33.196 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:03:33.196 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:03:33.197 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:03:33.197 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:03:33.423 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:03:33.498 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:33.497+0000 7f49bec23900 -1 osd.2 0 log_to_monitors true 2026-02-20T18:03:34.425 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:03:34.425 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:03:34.425 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:03:34.425 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:03:34.426 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:03:34.426 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:03:34.655 INFO:tasks.workunit.client.0.trial195.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2350231731,v1:127.0.0.1:6819/2350231731] [v2:127.0.0.1:6820/2350231731,v1:127.0.0.1:6821/2350231731] exists,up fb1bbc15-0a94-45be-befa-f2bab0935b32 2026-02-20T18:03:34.659 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:03:34.659 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:03:34.660 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:03:34.660 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:03:34.660 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-20T18:03:34.660 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:03:34.660 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:03:34.660 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-20T18:03:34.660 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:03:34.660 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-20T18:03:34.660 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-20T18:03:34.660 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:03:34.660 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:03:34.661 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:03:34.661 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-20T18:03:34.661 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-20T18:03:34.661 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:03:34.661 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:03:34.661 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:03:34.661 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:03:34.661 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:03:34.661 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:03:34.661 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:03:34.661 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:03:34.661 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:03:34.662 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:03:34.662 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:03:34.662 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:03:34.662 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:03:34.662 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:03:34.662 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:03:34.662 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:03:34.662 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:03:34.662 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:03:34.662 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:03:34.662 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:03:34.663 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:03:34.663 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:03:34.663 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-20T18:03:34.663 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:03:34.663 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c042619e-5ad2-48f5-b0f5-19a1d9a08a37 2026-02-20T18:03:34.663 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 c042619e-5ad2-48f5-b0f5-19a1d9a08a37' 2026-02-20T18:03:34.663 INFO:tasks.workunit.client.0.trial195.stdout:add osd3 c042619e-5ad2-48f5-b0f5-19a1d9a08a37 2026-02-20T18:03:34.664 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:03:34.673 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD2oZhpVgoDKBAAqFH6yizVv/Dpr5CgJLvNoA== 2026-02-20T18:03:34.673 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD2oZhpVgoDKBAAqFH6yizVv/Dpr5CgJLvNoA=="}' 2026-02-20T18:03:34.673 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c042619e-5ad2-48f5-b0f5-19a1d9a08a37 -i td/test-erasure-eio/3/new.json 2026-02-20T18:03:34.915 INFO:tasks.workunit.client.0.trial195.stdout:3 2026-02-20T18:03:34.923 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-20T18:03:34.924 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD2oZhpVgoDKBAAqFH6yizVv/Dpr5CgJLvNoA== --osd-uuid c042619e-5ad2-48f5-b0f5-19a1d9a08a37 2026-02-20T18:03:34.946 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:34.945+0000 7fd28756e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:34.947 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:34.946+0000 7fd28756e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:34.948 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:34.947+0000 7fd28756e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:34.949 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:34.948+0000 7fd28756e900 -1 bdev(0x55a73c888c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-20T18:03:34.949 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:34.948+0000 7fd28756e900 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-20T18:03:35.288 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-20T18:03:35.288 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:03:35.289 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-20T18:03:35.289 INFO:tasks.workunit.client.0.trial195.stdout:adding osd3 key to auth repository 2026-02-20T18:03:35.289 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:03:35.611 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-20T18:03:35.611 INFO:tasks.workunit.client.0.trial195.stdout:start osd.3 2026-02-20T18:03:35.612 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:03:35.612 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:03:35.612 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:03:35.613 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:03:35.630 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:35.629+0000 7fee2dd8c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:35.632 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:35.631+0000 7fee2dd8c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:35.633 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:35.632+0000 7fee2dd8c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:35.775 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:35.773+0000 7fee2dd8c900 -1 Falling back to public interface 2026-02-20T18:03:35.839 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-20T18:03:35.839 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:03:35.839 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-20T18:03:35.840 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:03:35.840 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:03:35.840 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:03:35.840 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:03:35.840 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:03:35.841 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:03:35.841 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T18:03:36.065 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:03:36.092 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:36.091+0000 7fee2dd8c900 -1 osd.3 0 log_to_monitors true 2026-02-20T18:03:37.066 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:03:37.066 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:03:37.066 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:03:37.066 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:03:37.066 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:03:37.066 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T18:03:37.300 INFO:tasks.workunit.client.0.trial195.stdout:osd.3 up in weight 1 up_from 26 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2776184323,v1:127.0.0.1:6827/2776184323] [v2:127.0.0.1:6828/2776184323,v1:127.0.0.1:6829/2776184323] exists,up c042619e-5ad2-48f5-b0f5-19a1d9a08a37 2026-02-20T18:03:37.303 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:03:37.303 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:03:37.303 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:03:37.303 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:03:37.303 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-20T18:03:37.304 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:03:37.304 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:03:37.304 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-20T18:03:37.304 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:03:37.304 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-20T18:03:37.304 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-20T18:03:37.304 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:03:37.304 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:03:37.304 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:03:37.304 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-20T18:03:37.304 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-20T18:03:37.304 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:03:37.304 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:03:37.304 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:03:37.304 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:03:37.304 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:03:37.305 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:03:37.305 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:03:37.305 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:03:37.305 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:03:37.305 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:03:37.305 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:03:37.305 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:03:37.305 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:03:37.305 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:03:37.305 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:03:37.305 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:03:37.305 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:03:37.305 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:03:37.305 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:03:37.306 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:03:37.306 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:03:37.306 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:03:37.306 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-20T18:03:37.306 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:03:37.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2ba5dc8f-230a-4f23-be41-deec70e79a51 2026-02-20T18:03:37.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 2ba5dc8f-230a-4f23-be41-deec70e79a51' 2026-02-20T18:03:37.307 INFO:tasks.workunit.client.0.trial195.stdout:add osd4 2ba5dc8f-230a-4f23-be41-deec70e79a51 2026-02-20T18:03:37.308 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:03:37.318 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD5oZhpiWbgEhAAFoEs0Q5PTzrnIDPkWlsn4Q== 2026-02-20T18:03:37.318 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD5oZhpiWbgEhAAFoEs0Q5PTzrnIDPkWlsn4Q=="}' 2026-02-20T18:03:37.318 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2ba5dc8f-230a-4f23-be41-deec70e79a51 -i td/test-erasure-eio/4/new.json 2026-02-20T18:03:37.549 INFO:tasks.workunit.client.0.trial195.stdout:4 2026-02-20T18:03:37.558 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-20T18:03:37.559 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD5oZhpiWbgEhAAFoEs0Q5PTzrnIDPkWlsn4Q== --osd-uuid 2ba5dc8f-230a-4f23-be41-deec70e79a51 2026-02-20T18:03:37.580 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:37.579+0000 7f349f097900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:37.581 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:37.580+0000 7f349f097900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:37.582 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:37.581+0000 7f349f097900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:37.582 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:37.581+0000 7f349f097900 -1 bdev(0x56195c812c00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-20T18:03:37.583 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:37.582+0000 7f349f097900 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-20T18:03:37.913 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-20T18:03:37.913 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:03:37.914 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-20T18:03:37.914 INFO:tasks.workunit.client.0.trial195.stdout:adding osd4 key to auth repository 2026-02-20T18:03:37.915 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:03:38.232 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-20T18:03:38.233 INFO:tasks.workunit.client.0.trial195.stdout:start osd.4 2026-02-20T18:03:38.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:03:38.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:03:38.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:03:38.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:03:38.251 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:38.250+0000 7f8154472900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:38.253 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:38.252+0000 7f8154472900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:38.253 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:38.253+0000 7f8154472900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:38.384 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:38.383+0000 7f8154472900 -1 Falling back to public interface 2026-02-20T18:03:38.458 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-20T18:03:38.458 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:03:38.458 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-20T18:03:38.458 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:03:38.458 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:03:38.458 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:03:38.458 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:03:38.459 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:03:38.459 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:03:38.459 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-20T18:03:38.681 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:38.680+0000 7f8154472900 -1 osd.4 0 log_to_monitors true 2026-02-20T18:03:38.684 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:03:39.686 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:03:39.686 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:03:39.686 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:03:39.687 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:03:39.687 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:03:39.687 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-20T18:03:39.919 INFO:tasks.workunit.client.0.trial195.stdout:osd.4 up in weight 1 up_from 34 up_thru 35 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/345089865,v1:127.0.0.1:6835/345089865] [v2:127.0.0.1:6836/345089865,v1:127.0.0.1:6837/345089865] exists,up 2ba5dc8f-230a-4f23-be41-deec70e79a51 2026-02-20T18:03:39.923 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:03:39.923 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:03:39.923 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:03:39.923 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:03:39.923 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-20T18:03:39.924 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:03:39.924 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:03:39.924 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-20T18:03:39.924 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:03:39.924 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-20T18:03:39.924 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-20T18:03:39.924 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:03:39.924 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:03:39.924 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:03:39.924 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-20T18:03:39.924 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-20T18:03:39.924 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:03:39.925 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:03:39.925 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:03:39.925 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:03:39.925 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:03:39.925 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:03:39.925 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:03:39.925 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:03:39.925 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:03:39.925 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:03:39.925 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:03:39.925 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:03:39.925 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:03:39.926 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:03:39.926 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:03:39.926 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:03:39.926 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:03:39.926 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:03:39.926 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:03:39.926 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:03:39.926 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:03:39.926 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:03:39.926 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-20T18:03:39.926 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:03:39.926 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f60e2627-7620-471b-9595-5b879e823e13 2026-02-20T18:03:39.926 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 f60e2627-7620-471b-9595-5b879e823e13' 2026-02-20T18:03:39.927 INFO:tasks.workunit.client.0.trial195.stdout:add osd5 f60e2627-7620-471b-9595-5b879e823e13 2026-02-20T18:03:39.927 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:03:39.937 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD7oZhp1tXLNxAAyILedpr9WktiyD6y+p8N3A== 2026-02-20T18:03:39.938 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD7oZhp1tXLNxAAyILedpr9WktiyD6y+p8N3A=="}' 2026-02-20T18:03:39.938 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f60e2627-7620-471b-9595-5b879e823e13 -i td/test-erasure-eio/5/new.json 2026-02-20T18:03:40.171 INFO:tasks.workunit.client.0.trial195.stdout:5 2026-02-20T18:03:40.178 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-20T18:03:40.180 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD7oZhp1tXLNxAAyILedpr9WktiyD6y+p8N3A== --osd-uuid f60e2627-7620-471b-9595-5b879e823e13 2026-02-20T18:03:40.200 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:40.199+0000 7f83a54f7900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:40.201 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:40.201+0000 7f83a54f7900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:40.202 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:40.201+0000 7f83a54f7900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:40.202 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:40.202+0000 7f83a54f7900 -1 bdev(0x55b02cd14c00 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-20T18:03:40.203 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:40.202+0000 7f83a54f7900 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-20T18:03:40.526 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-20T18:03:40.527 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:03:40.527 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-20T18:03:40.528 INFO:tasks.workunit.client.0.trial195.stdout:adding osd5 key to auth repository 2026-02-20T18:03:40.528 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:03:40.848 INFO:tasks.workunit.client.0.trial195.stdout:start osd.5 2026-02-20T18:03:40.849 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-20T18:03:40.849 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:03:40.849 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:03:40.849 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:03:40.849 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:03:40.868 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:40.867+0000 7f9535d4d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:40.869 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:40.868+0000 7f9535d4d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:40.870 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:40.869+0000 7f9535d4d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:40.984 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:40.983+0000 7f9535d4d900 -1 Falling back to public interface 2026-02-20T18:03:41.075 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-20T18:03:41.075 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:03:41.075 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-20T18:03:41.075 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:03:41.075 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:03:41.075 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:03:41.075 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:03:41.075 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:03:41.075 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:03:41.075 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-20T18:03:41.285 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:41.284+0000 7f9535d4d900 -1 osd.5 0 log_to_monitors true 2026-02-20T18:03:41.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:03:42.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:03:42.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:03:42.309 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:03:42.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:03:42.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:03:42.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-20T18:03:42.533 INFO:tasks.workunit.client.0.trial195.stdout:osd.5 up in weight 1 up_from 43 up_thru 44 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/1760440924,v1:127.0.0.1:6843/1760440924] [v2:127.0.0.1:6844/1760440924,v1:127.0.0.1:6845/1760440924] exists,up f60e2627-7620-471b-9595-5b879e823e13 2026-02-20T18:03:42.537 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:03:42.538 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:03:42.538 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:03:42.538 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:03:42.538 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-20T18:03:42.538 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:03:42.538 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:03:42.538 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-20T18:03:42.538 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:03:42.538 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-20T18:03:42.538 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-20T18:03:42.538 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:03:42.539 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:03:42.539 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:03:42.539 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-20T18:03:42.539 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-20T18:03:42.539 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:03:42.539 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:03:42.539 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:03:42.539 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:03:42.539 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:03:42.539 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:03:42.539 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:03:42.539 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:03:42.540 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:03:42.540 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:03:42.540 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:03:42.540 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:03:42.540 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:03:42.540 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:03:42.540 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:03:42.540 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:03:42.540 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:03:42.540 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:03:42.540 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:03:42.540 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:03:42.541 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:03:42.541 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:03:42.541 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-20T18:03:42.541 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:03:42.541 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2ac643dc-f7ce-4dda-93c0-fe92b88a81c9 2026-02-20T18:03:42.541 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 2ac643dc-f7ce-4dda-93c0-fe92b88a81c9' 2026-02-20T18:03:42.541 INFO:tasks.workunit.client.0.trial195.stdout:add osd6 2ac643dc-f7ce-4dda-93c0-fe92b88a81c9 2026-02-20T18:03:42.542 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:03:42.551 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD+oZhpgbPDIBAAmG+WGfAMTCG7/smEpft4ng== 2026-02-20T18:03:42.551 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD+oZhpgbPDIBAAmG+WGfAMTCG7/smEpft4ng=="}' 2026-02-20T18:03:42.551 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2ac643dc-f7ce-4dda-93c0-fe92b88a81c9 -i td/test-erasure-eio/6/new.json 2026-02-20T18:03:42.778 INFO:tasks.workunit.client.0.trial195.stdout:6 2026-02-20T18:03:42.785 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-20T18:03:42.786 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD+oZhpgbPDIBAAmG+WGfAMTCG7/smEpft4ng== --osd-uuid 2ac643dc-f7ce-4dda-93c0-fe92b88a81c9 2026-02-20T18:03:42.807 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:42.806+0000 7fa4872f3900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:42.808 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:42.808+0000 7fa4872f3900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:42.809 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:42.809+0000 7fa4872f3900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:42.810 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:42.809+0000 7fa4872f3900 -1 bdev(0x556776102c00 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-20T18:03:42.810 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:42.809+0000 7fa4872f3900 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-20T18:03:43.138 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-20T18:03:43.138 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:03:43.139 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-20T18:03:43.139 INFO:tasks.workunit.client.0.trial195.stdout:adding osd6 key to auth repository 2026-02-20T18:03:43.139 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:03:43.459 INFO:tasks.workunit.client.0.trial195.stdout:start osd.6 2026-02-20T18:03:43.460 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-20T18:03:43.460 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:03:43.460 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:03:43.460 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:03:43.460 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:03:43.479 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:43.478+0000 7fa057ead900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:43.481 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:43.480+0000 7fa057ead900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:43.482 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:43.481+0000 7fa057ead900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:03:43.623 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:43.622+0000 7fa057ead900 -1 Falling back to public interface 2026-02-20T18:03:43.686 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-20T18:03:43.686 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:03:43.686 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-20T18:03:43.686 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:03:43.686 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:03:43.686 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:03:43.686 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:03:43.686 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:03:43.686 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:03:43.686 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-20T18:03:43.913 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:03:43.976 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:03:43.976+0000 7fa057ead900 -1 osd.6 0 log_to_monitors true 2026-02-20T18:03:44.914 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:03:44.915 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:03:44.915 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:03:44.916 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:03:44.916 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:03:44.916 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-20T18:03:45.140 INFO:tasks.workunit.client.0.trial195.stdout:osd.6 up in weight 1 up_from 49 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/2909253259,v1:127.0.0.1:6851/2909253259] [v2:127.0.0.1:6852/2909253259,v1:127.0.0.1:6853/2909253259] exists,up 2ac643dc-f7ce-4dda-93c0-fe92b88a81c9 2026-02-20T18:03:45.142 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:03:45.142 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:03:45.142 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:03:45.142 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-20T18:03:45.142 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-20T18:03:45.143 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-20T18:03:45.143 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:03:45.143 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:03:45.143 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:03:45.143 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.0.asok 2026-02-20T18:03:45.143 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-20T18:03:45.143 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-osd.0.asok log flush 2026-02-20T18:03:45.188 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T18:03:45.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-20T18:03:45.194 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T18:03:28.225+0000 7f18f92ad900 0 load: jerasure load: lrc 2026-02-20T18:03:45.195 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:506: TEST_ec_recovery_multiple_objects_eio: CEPH_ARGS='--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:03:45.195 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:508: TEST_ec_recovery_multiple_objects_eio: local poolname=pool-jerasure 2026-02-20T18:03:45.195 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:509: TEST_ec_recovery_multiple_objects_eio: create_erasure_coded_pool pool-jerasure 3 2 2026-02-20T18:03:45.195 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-20T18:03:45.196 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-20T18:03:45.196 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-20T18:03:45.196 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-20T18:03:45.196 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-20T18:03:45.196 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-20T18:03:45.196 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-20T18:03:45.483 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-20T18:03:45.483 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-20T18:03:45.854 INFO:tasks.workunit.client.0.trial195.stderr:pool 'pool-jerasure' already exists 2026-02-20T18:03:45.862 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-20T18:03:46.864 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-20T18:03:46.865 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:03:46.865 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:03:46.865 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:03:46.865 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:03:46.866 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:03:46.866 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:03:46.866 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:03:46.866 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:03:46.866 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:03:46.950 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:03:46.950 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:03:46.950 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:03:46.950 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:03:46.950 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:03:46.951 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:03:47.180 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:03:47.181 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:03:47.181 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:03:47.181 INFO:tasks.workunit.client.0.trial195.stderr:3 2026-02-20T18:03:47.181 INFO:tasks.workunit.client.0.trial195.stderr:4 2026-02-20T18:03:47.181 INFO:tasks.workunit.client.0.trial195.stderr:5 2026-02-20T18:03:47.181 INFO:tasks.workunit.client.0.trial195.stderr:6' 2026-02-20T18:03:47.181 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:03:47.181 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:03:47.181 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:03:47.244 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803782 2026-02-20T18:03:47.244 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803782 2026-02-20T18:03:47.244 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782' 2026-02-20T18:03:47.244 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:03:47.244 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:03:47.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574854 2026-02-20T18:03:47.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574854 2026-02-20T18:03:47.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-55834574854' 2026-02-20T18:03:47.310 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:03:47.310 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:03:47.374 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411333 2026-02-20T18:03:47.374 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411333 2026-02-20T18:03:47.374 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-77309411333' 2026-02-20T18:03:47.375 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:03:47.375 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:03:47.441 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149701 2026-02-20T18:03:47.441 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149701 2026-02-20T18:03:47.441 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-77309411333 3-111669149701' 2026-02-20T18:03:47.441 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:03:47.441 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-20T18:03:47.506 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888068 2026-02-20T18:03:47.506 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888068 2026-02-20T18:03:47.506 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-77309411333 3-111669149701 4-146028888068' 2026-02-20T18:03:47.506 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:03:47.506 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-20T18:03:47.574 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593732 2026-02-20T18:03:47.574 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593732 2026-02-20T18:03:47.574 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-77309411333 3-111669149701 4-146028888068 5-184683593732' 2026-02-20T18:03:47.575 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:03:47.575 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-20T18:03:47.641 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=210453397507 2026-02-20T18:03:47.641 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 210453397507 2026-02-20T18:03:47.641 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-77309411333 3-111669149701 4-146028888068 5-184683593732 6-210453397507' 2026-02-20T18:03:47.641 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:03:47.641 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803782 2026-02-20T18:03:47.641 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:03:47.642 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:03:47.642 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803782 2026-02-20T18:03:47.642 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:03:47.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803782 2026-02-20T18:03:47.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803782' 2026-02-20T18:03:47.644 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.0 seq 25769803782 2026-02-20T18:03:47.644 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:03:47.865 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803780 -lt 25769803782 2026-02-20T18:03:47.866 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:03:48.866 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:03:48.867 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:03:49.093 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803782 -lt 25769803782 2026-02-20T18:03:49.094 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:03:49.094 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574854 2026-02-20T18:03:49.095 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:03:49.095 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:03:49.096 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-55834574854 2026-02-20T18:03:49.096 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:03:49.097 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574854 2026-02-20T18:03:49.097 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 55834574854' 2026-02-20T18:03:49.097 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.1 seq 55834574854 2026-02-20T18:03:49.098 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:03:49.323 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574854 -lt 55834574854 2026-02-20T18:03:49.323 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:03:49.324 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411333 2026-02-20T18:03:49.324 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:03:49.325 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:03:49.325 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-77309411333 2026-02-20T18:03:49.326 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:03:49.326 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=77309411333 2026-02-20T18:03:49.326 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 77309411333' 2026-02-20T18:03:49.327 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.2 seq 77309411333 2026-02-20T18:03:49.327 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:03:49.550 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411333 -lt 77309411333 2026-02-20T18:03:49.550 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:03:49.551 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149701 2026-02-20T18:03:49.551 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:03:49.552 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:03:49.552 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149701 2026-02-20T18:03:49.552 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:03:49.553 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149701 2026-02-20T18:03:49.554 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.3 seq 111669149701 2026-02-20T18:03:49.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149701' 2026-02-20T18:03:49.554 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:03:49.778 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149701 -lt 111669149701 2026-02-20T18:03:49.779 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:03:49.779 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-146028888068 2026-02-20T18:03:49.779 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:03:49.780 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-20T18:03:49.780 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-146028888068 2026-02-20T18:03:49.781 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:03:49.781 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888068 2026-02-20T18:03:49.782 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 146028888068' 2026-02-20T18:03:49.782 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.4 seq 146028888068 2026-02-20T18:03:49.782 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-20T18:03:50.005 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888068 -lt 146028888068 2026-02-20T18:03:50.005 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:03:50.006 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-184683593732 2026-02-20T18:03:50.006 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:03:50.007 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-20T18:03:50.007 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-184683593732 2026-02-20T18:03:50.007 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:03:50.009 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=184683593732 2026-02-20T18:03:50.009 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 184683593732' 2026-02-20T18:03:50.009 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.5 seq 184683593732 2026-02-20T18:03:50.009 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-20T18:03:50.239 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 184683593732 -lt 184683593732 2026-02-20T18:03:50.239 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:03:50.239 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-210453397507 2026-02-20T18:03:50.239 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:03:50.240 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-20T18:03:50.241 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-210453397507 2026-02-20T18:03:50.241 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:03:50.242 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=210453397507 2026-02-20T18:03:50.242 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 210453397507' 2026-02-20T18:03:50.242 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.6 seq 210453397507 2026-02-20T18:03:50.242 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-20T18:03:50.466 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 210453397507 -lt 210453397507 2026-02-20T18:03:50.467 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:03:50.467 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:03:50.467 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:03:50.781 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:03:50.781 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:03:50.781 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:03:50.781 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:03:50.781 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:03:50.781 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:03:50.782 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:03:50.782 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:03:51.002 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:03:51.002 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:03:51.003 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:03:51.003 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:03:51.326 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:03:51.326 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:03:51.326 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:03:51.326 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:511: TEST_ec_recovery_multiple_objects_eio: rados_put td/test-erasure-eio pool-jerasure test1 2026-02-20T18:03:51.326 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-20T18:03:51.326 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-20T18:03:51.327 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test1 2026-02-20T18:03:51.327 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:03:51.327 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-20T18:03:51.327 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:03:51.327 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-20T18:03:51.327 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:03:51.327 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-20T18:03:51.327 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:03:51.327 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-20T18:03:51.327 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test1 td/test-erasure-eio/ORIGINAL 2026-02-20T18:03:51.346 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:512: TEST_ec_recovery_multiple_objects_eio: rados_put td/test-erasure-eio pool-jerasure test2 2026-02-20T18:03:51.346 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-20T18:03:51.346 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-20T18:03:51.346 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test2 2026-02-20T18:03:51.346 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:03:51.346 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-20T18:03:51.346 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:03:51.346 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-20T18:03:51.346 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:03:51.347 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-20T18:03:51.347 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:03:51.347 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-20T18:03:51.347 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test2 td/test-erasure-eio/ORIGINAL 2026-02-20T18:03:51.366 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:513: TEST_ec_recovery_multiple_objects_eio: rados_put td/test-erasure-eio pool-jerasure test3 2026-02-20T18:03:51.366 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-20T18:03:51.366 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-20T18:03:51.366 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test3 2026-02-20T18:03:51.366 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:03:51.366 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-20T18:03:51.366 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:03:51.366 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-20T18:03:51.367 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:03:51.367 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-20T18:03:51.367 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:03:51.367 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-20T18:03:51.367 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test3 td/test-erasure-eio/ORIGINAL 2026-02-20T18:03:51.387 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:516: TEST_ec_recovery_multiple_objects_eio: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 0 2026-02-20T18:03:51.387 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=ec 2026-02-20T18:03:51.387 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-20T18:03:51.388 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=data 2026-02-20T18:03:51.388 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-20T18:03:51.388 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=pool-jerasure 2026-02-20T18:03:51.388 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-20T18:03:51.388 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=myobject 2026-02-20T18:03:51.388 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-20T18:03:51.389 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/test-erasure-eio 2026-02-20T18:03:51.389 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-20T18:03:51.389 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=0 2026-02-20T18:03:51.389 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-20T18:03:51.389 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds pool-jerasure myobject 2026-02-20T18:03:51.389 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:03:51.390 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-20T18:03:51.390 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-20T18:03:51.390 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:03:51.614 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T18:03:51.614 INFO:tasks.workunit.client.0.trial195.stderr:5 2026-02-20T18:03:51.614 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:03:51.614 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T18:03:51.614 INFO:tasks.workunit.client.0.trial195.stderr:6' 2026-02-20T18:03:51.614 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2026-02-20T18:03:51.615 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('3' '5' '1' '0' '6') 2026-02-20T18:03:51.615 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-20T18:03:51.615 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=3 2026-02-20T18:03:51.615 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' ec '!=' ec ']' 2026-02-20T18:03:51.615 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/test-erasure-eio/3/type 2026-02-20T18:03:51.617 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-20T18:03:51.617 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-20T18:03:51.617 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-20T18:03:51.617 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-20T18:03:51.617 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-20T18:03:51.617 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-20T18:03:51.617 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-20T18:03:51.617 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-20T18:03:51.618 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-20T18:03:51.618 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-20T18:03:51.618 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:03:51.618 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:03:51.618 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:03:51.618 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.3.asok 2026-02-20T18:03:51.618 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.85808/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-20T18:03:51.666 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-20T18:03:51.667 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-20T18:03:51.668 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-20T18:03:51.668 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.3 2026-02-20T18:03:51.668 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-20T18:03:51.668 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-20T18:03:51.668 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:03:51.668 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:03:51.669 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:03:51.669 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.3.asok 2026-02-20T18:03:51.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-20T18:03:51.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-osd.3.asok injectdataerr pool-jerasure myobject 0 2026-02-20T18:03:51.717 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:517: TEST_ec_recovery_multiple_objects_eio: ceph osd out 0 2026-02-20T18:03:51.991 INFO:tasks.workunit.client.0.trial195.stderr:osd.0 is already out. 2026-02-20T18:03:52.001 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:520: TEST_ec_recovery_multiple_objects_eio: wait_for_clean 2026-02-20T18:03:52.001 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:03:52.001 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:03:52.001 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:03:52.001 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:03:52.001 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:03:52.001 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:03:52.002 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:03:52.002 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:03:52.002 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:03:52.087 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:03:52.087 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:03:52.088 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:03:52.088 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:03:52.088 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:03:52.088 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:03:52.311 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:03:52.311 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:03:52.311 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:03:52.311 INFO:tasks.workunit.client.0.trial195.stderr:3 2026-02-20T18:03:52.312 INFO:tasks.workunit.client.0.trial195.stderr:4 2026-02-20T18:03:52.312 INFO:tasks.workunit.client.0.trial195.stderr:5 2026-02-20T18:03:52.312 INFO:tasks.workunit.client.0.trial195.stderr:6' 2026-02-20T18:03:52.312 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:03:52.312 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:03:52.312 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:03:52.377 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803785 2026-02-20T18:03:52.377 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803785 2026-02-20T18:03:52.377 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803785' 2026-02-20T18:03:52.377 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:03:52.377 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:03:52.440 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574857 2026-02-20T18:03:52.441 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574857 2026-02-20T18:03:52.441 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803785 1-55834574857' 2026-02-20T18:03:52.441 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:03:52.441 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:03:52.504 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411336 2026-02-20T18:03:52.504 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411336 2026-02-20T18:03:52.505 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803785 1-55834574857 2-77309411336' 2026-02-20T18:03:52.505 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:03:52.505 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:03:52.569 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149704 2026-02-20T18:03:52.570 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149704 2026-02-20T18:03:52.570 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803785 1-55834574857 2-77309411336 3-111669149704' 2026-02-20T18:03:52.570 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:03:52.570 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-20T18:03:52.633 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888071 2026-02-20T18:03:52.633 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888071 2026-02-20T18:03:52.633 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803785 1-55834574857 2-77309411336 3-111669149704 4-146028888071' 2026-02-20T18:03:52.634 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:03:52.634 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-20T18:03:52.698 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593735 2026-02-20T18:03:52.699 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593735 2026-02-20T18:03:52.699 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803785 1-55834574857 2-77309411336 3-111669149704 4-146028888071 5-184683593735' 2026-02-20T18:03:52.699 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:03:52.699 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-20T18:03:52.763 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=210453397510 2026-02-20T18:03:52.763 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 210453397510 2026-02-20T18:03:52.763 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803785 1-55834574857 2-77309411336 3-111669149704 4-146028888071 5-184683593735 6-210453397510' 2026-02-20T18:03:52.763 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:03:52.764 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803785 2026-02-20T18:03:52.764 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:03:52.764 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:03:52.765 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803785 2026-02-20T18:03:52.765 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:03:52.765 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803785 2026-02-20T18:03:52.765 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803785' 2026-02-20T18:03:52.766 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.0 seq 25769803785 2026-02-20T18:03:52.766 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:03:52.992 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 0 -lt 25769803785 2026-02-20T18:03:52.992 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:03:53.993 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:03:53.993 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:03:54.217 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803785 -lt 25769803785 2026-02-20T18:03:54.217 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:03:54.218 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574857 2026-02-20T18:03:54.218 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:03:54.219 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:03:54.220 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-55834574857 2026-02-20T18:03:54.220 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:03:54.221 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574857 2026-02-20T18:03:54.221 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 55834574857' 2026-02-20T18:03:54.222 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.1 seq 55834574857 2026-02-20T18:03:54.222 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:03:54.443 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574857 -lt 55834574857 2026-02-20T18:03:54.444 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:03:54.444 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411336 2026-02-20T18:03:54.445 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:03:54.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:03:54.446 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-77309411336 2026-02-20T18:03:54.446 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:03:54.447 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=77309411336 2026-02-20T18:03:54.447 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 77309411336' 2026-02-20T18:03:54.448 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.2 seq 77309411336 2026-02-20T18:03:54.448 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:03:54.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411336 -lt 77309411336 2026-02-20T18:03:54.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:03:54.670 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149704 2026-02-20T18:03:54.670 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:03:54.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:03:54.671 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149704 2026-02-20T18:03:54.671 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:03:54.673 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149704 2026-02-20T18:03:54.673 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149704' 2026-02-20T18:03:54.673 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.3 seq 111669149704 2026-02-20T18:03:54.674 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:03:54.896 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149704 -lt 111669149704 2026-02-20T18:03:54.896 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:03:54.896 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-146028888071 2026-02-20T18:03:54.896 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:03:54.897 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-20T18:03:54.898 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-146028888071 2026-02-20T18:03:54.898 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:03:54.899 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888071 2026-02-20T18:03:54.899 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 146028888071' 2026-02-20T18:03:54.899 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.4 seq 146028888071 2026-02-20T18:03:54.900 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-20T18:03:55.122 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888071 -lt 146028888071 2026-02-20T18:03:55.122 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:03:55.122 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-184683593735 2026-02-20T18:03:55.123 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:03:55.123 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-20T18:03:55.124 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-184683593735 2026-02-20T18:03:55.124 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:03:55.125 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=184683593735 2026-02-20T18:03:55.125 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 184683593735' 2026-02-20T18:03:55.125 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.5 seq 184683593735 2026-02-20T18:03:55.125 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-20T18:03:55.349 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 184683593735 -lt 184683593735 2026-02-20T18:03:55.349 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:03:55.350 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-210453397510 2026-02-20T18:03:55.350 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:03:55.351 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-20T18:03:55.351 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-210453397510 2026-02-20T18:03:55.351 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:03:55.353 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=210453397510 2026-02-20T18:03:55.353 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 210453397510' 2026-02-20T18:03:55.353 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.6 seq 210453397510 2026-02-20T18:03:55.353 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-20T18:03:55.579 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 210453397510 -lt 210453397510 2026-02-20T18:03:55.580 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:03:55.580 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:03:55.580 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:03:55.892 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:03:55.892 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:03:55.892 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:03:55.892 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:03:55.892 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:03:55.892 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:03:55.892 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:03:55.893 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:03:56.112 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-20T18:03:56.112 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:03:56.112 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:03:56.112 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:03:56.421 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-20T18:03:56.421 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' -1 2026-02-20T18:03:56.422 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-20T18:03:56.422 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=4 2026-02-20T18:03:56.422 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-20T18:03:56.422 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-20T18:03:56.523 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-20T18:03:56.524 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:03:56.524 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:03:56.524 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:03:56.524 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:03:56.524 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:03:56.524 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:03:56.525 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:03:56.747 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-20T18:03:56.747 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:03:56.748 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:03:56.748 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:03:57.061 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-20T18:03:57.061 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' 4 2026-02-20T18:03:57.061 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-20T18:03:57.061 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-20T18:03:57.061 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-20T18:03:57.061 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-20T18:03:57.061 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-20T18:03:57.062 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-20T18:03:57.062 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-20T18:03:57.376 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-20T18:03:57.377 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-20T18:03:57.377 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 13 )) 2026-02-20T18:03:57.377 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-20T18:03:57.377 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-20T18:03:57.576 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-20T18:03:57.577 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:03:57.577 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:03:57.577 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:03:57.577 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:03:57.577 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:03:57.577 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:03:57.577 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:03:57.802 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-20T18:03:57.802 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:03:57.803 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:03:57.803 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:03:58.114 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-20T18:03:58.115 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' 4 2026-02-20T18:03:58.115 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-20T18:03:58.115 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-20T18:03:58.115 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-20T18:03:58.115 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-20T18:03:58.115 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-20T18:03:58.115 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-20T18:03:58.115 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-20T18:03:58.436 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-20T18:03:58.436 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-20T18:03:58.436 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 2 >= 13 )) 2026-02-20T18:03:58.436 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-20T18:03:58.437 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.4 2026-02-20T18:03:58.837 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-20T18:03:58.837 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:03:58.838 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:03:58.838 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:03:58.838 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:03:58.838 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:03:58.838 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:03:58.838 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:03:59.064 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:03:59.065 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:03:59.065 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:03:59.066 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:03:59.375 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:03:59.375 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:03:59.375 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:03:59.376 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:522: TEST_ec_recovery_multiple_objects_eio: rados_get td/test-erasure-eio pool-jerasure test1 2026-02-20T18:03:59.376 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-20T18:03:59.376 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-20T18:03:59.376 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test1 2026-02-20T18:03:59.376 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-20T18:03:59.376 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-20T18:03:59.376 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test1 td/test-erasure-eio/COPY 2026-02-20T18:03:59.396 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-20T18:03:59.396 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-20T18:03:59.397 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:523: TEST_ec_recovery_multiple_objects_eio: rados_get td/test-erasure-eio pool-jerasure test2 2026-02-20T18:03:59.398 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-20T18:03:59.398 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-20T18:03:59.398 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test2 2026-02-20T18:03:59.398 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-20T18:03:59.398 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-20T18:03:59.398 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test2 td/test-erasure-eio/COPY 2026-02-20T18:03:59.416 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-20T18:03:59.416 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-20T18:03:59.418 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:524: TEST_ec_recovery_multiple_objects_eio: rados_get td/test-erasure-eio pool-jerasure test3 2026-02-20T18:03:59.418 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-20T18:03:59.418 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-20T18:03:59.418 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test3 2026-02-20T18:03:59.418 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-20T18:03:59.419 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-20T18:03:59.419 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test3 td/test-erasure-eio/COPY 2026-02-20T18:03:59.436 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-20T18:03:59.437 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-20T18:03:59.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:526: TEST_ec_recovery_multiple_objects_eio: delete_erasure_coded_pool pool-jerasure 2026-02-20T18:03:59.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-20T18:03:59.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-20T18:03:59.710 INFO:tasks.workunit.client.0.trial195.stderr:pool 'pool-jerasure' does not exist 2026-02-20T18:03:59.718 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-20T18:03:59.995 INFO:tasks.workunit.client.0.trial195.stderr:erasure-code-profile myprofile does not exist 2026-02-20T18:04:00.002 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-20T18:04:00.002 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-20T18:04:00.002 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T18:04:00.002 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-20T18:04:00.003 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:04:00.003 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:04:00.003 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:04:00.003 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:04:00.003 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:04:00.113 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:04:00.114 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T18:04:00.114 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T18:04:00.115 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T18:04:00.116 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T18:04:00.117 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T18:04:00.117 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T18:04:00.117 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:04:00.118 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T18:04:00.118 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T18:04:00.118 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:04:00.119 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T18:04:00.120 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T18:04:00.120 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-20T18:04:00.135 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T18:04:00.135 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:04:00.135 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:04:00.136 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.85808 2026-02-20T18:04:00.137 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T18:04:00.137 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T18:04:00.137 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-20T18:04:00.138 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-20T18:04:00.138 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-20T18:04:00.138 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-20T18:04:00.138 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-20T18:04:00.138 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T18:04:00.138 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-20T18:04:00.138 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:04:00.138 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:04:00.138 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:04:00.138 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:04:00.138 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:04:00.139 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:04:00.139 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T18:04:00.139 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T18:04:00.140 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T18:04:00.141 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T18:04:00.141 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T18:04:00.141 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T18:04:00.142 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:04:00.142 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T18:04:00.142 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T18:04:00.143 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:04:00.143 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T18:04:00.144 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T18:04:00.144 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-20T18:04:00.146 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T18:04:00.146 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:04:00.146 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:04:00.146 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.85808 2026-02-20T18:04:00.147 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T18:04:00.147 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T18:04:00.147 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-20T18:04:00.148 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-20T18:04:00.148 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:04:00.148 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:04:00.148 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.85808 2026-02-20T18:04:00.149 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-20T18:04:00.150 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-20T18:04:00.150 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-20T18:04:00.150 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-20T18:04:00.150 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-20T18:04:00.151 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-20T18:04:00.151 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-20T18:04:00.151 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-20T18:04:00.151 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-20T18:04:00.151 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-20T18:04:00.151 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-20T18:04:00.172 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-20T18:04:00.172 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:04:00.172 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:04:00.172 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:04:00.172 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:04:00.172 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:04:00.172 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:04:00.173 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-20T18:04:00.199 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-20T18:04:00.200 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-20T18:04:00.201 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T18:04:00.201 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T18:04:00.201 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-20T18:04:00.201 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-20T18:04:00.201 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T18:04:00.201 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:04:00.201 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:04:00.202 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:04:00.203 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:04:00.203 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:04:00.203 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:04:00.203 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T18:04:00.203 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.85808/ceph-mon.a.asok config get fsid 2026-02-20T18:04:00.247 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-20T18:04:00.248 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T18:04:00.248 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T18:04:00.248 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-20T18:04:00.248 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-20T18:04:00.249 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T18:04:00.249 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:04:00.249 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:04:00.250 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:04:00.250 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:04:00.250 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:04:00.250 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:04:00.250 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T18:04:00.250 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.85808/ceph-mon.a.asok config get mon_host 2026-02-20T18:04:00.293 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-20T18:04:00.293 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-20T18:04:00.293 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-20T18:04:00.293 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-20T18:04:00.293 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-20T18:04:00.293 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-20T18:04:00.293 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-20T18:04:00.397 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-20T18:04:00.397 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:04:00.397 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:04:00.397 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:04:00.397 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:04:00.398 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:04:00.398 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:04:00.398 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T18:04:00.398 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T18:04:00.415 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-20T18:04:00.415 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-20T18:04:00.511 INFO:tasks.workunit.client.0.trial195.stderr:pool 'rbd' already exists 2026-02-20T18:04:00.517 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-20T18:04:01.519 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-20T18:04:01.520 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:04:01.520 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:04:01.520 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:04:01.520 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:04:01.520 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:04:01.520 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:04:01.520 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-20T18:04:01.520 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-mon.a.asok log flush 2026-02-20T18:04:01.561 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T18:04:01.564 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-20T18:04:01.566 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T18:04:00.190+0000 7f743ec97d80 0 load: jerasure load: lrc 2026-02-20T18:04:01.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_recovery_unfound td/test-erasure-eio 2026-02-20T18:04:01.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:629: TEST_ec_recovery_unfound: local dir=td/test-erasure-eio 2026-02-20T18:04:01.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:630: TEST_ec_recovery_unfound: local objname=myobject 2026-02-20T18:04:01.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:631: TEST_ec_recovery_unfound: local lastobj=100 2026-02-20T18:04:01.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:633: TEST_ec_recovery_unfound: local testobj=obj75 2026-02-20T18:04:01.568 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:635: TEST_ec_recovery_unfound: ORIG_ARGS='--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:04:01.568 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:636: TEST_ec_recovery_unfound: CEPH_ARGS+=' --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-20T18:04:01.568 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: CEPH_ARGS+=' --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-20T18:04:01.568 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:638: TEST_ec_recovery_unfound: setup_osds 5 2026-02-20T18:04:01.568 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=5 2026-02-20T18:04:01.568 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-20T18:04:01.568 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 5 - 1 2026-02-20T18:04:01.572 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 4 2026-02-20T18:04:01.572 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:04:01.572 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-20T18:04:01.572 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:04:01.572 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:04:01.572 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-20T18:04:01.572 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:04:01.572 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-20T18:04:01.573 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-20T18:04:01.573 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:04:01.573 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:04:01.573 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:04:01.573 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-20T18:04:01.573 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-20T18:04:01.573 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:04:01.573 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:04:01.573 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:04:01.573 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:04:01.573 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:04:01.573 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:04:01.574 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:04:01.574 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:04:01.574 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:04:01.574 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:04:01.574 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:04:01.574 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:04:01.574 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:04:01.574 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:04:01.574 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:04:01.574 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:04:01.574 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:04:01.574 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:04:01.574 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:04:01.575 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:04:01.575 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:04:01.575 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:04:01.575 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-20T18:04:01.575 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:04:01.575 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c320cfec-a245-4205-990c-5665f9055a37 2026-02-20T18:04:01.575 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 c320cfec-a245-4205-990c-5665f9055a37' 2026-02-20T18:04:01.575 INFO:tasks.workunit.client.0.trial195.stdout:add osd0 c320cfec-a245-4205-990c-5665f9055a37 2026-02-20T18:04:01.576 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:04:01.587 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQARophp9IvxIhAAzaCy2owiWg7f4nXrtk9RVQ== 2026-02-20T18:04:01.588 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQARophp9IvxIhAAzaCy2owiWg7f4nXrtk9RVQ=="}' 2026-02-20T18:04:01.588 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c320cfec-a245-4205-990c-5665f9055a37 -i td/test-erasure-eio/0/new.json 2026-02-20T18:04:01.696 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:04:01.702 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-20T18:04:01.704 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQARophp9IvxIhAAzaCy2owiWg7f4nXrtk9RVQ== --osd-uuid c320cfec-a245-4205-990c-5665f9055a37 2026-02-20T18:04:01.726 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:01.724+0000 7f83c5822900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:01.728 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:01.726+0000 7f83c5822900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:01.729 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:01.727+0000 7f83c5822900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:01.729 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:01.727+0000 7f83c5822900 -1 bdev(0x564526df4c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-20T18:04:01.730 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:01.727+0000 7f83c5822900 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-20T18:04:02.141 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-20T18:04:02.141 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:04:02.142 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-20T18:04:02.142 INFO:tasks.workunit.client.0.trial195.stdout:adding osd0 key to auth repository 2026-02-20T18:04:02.142 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:04:02.251 INFO:tasks.workunit.client.0.trial195.stdout:start osd.0 2026-02-20T18:04:02.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-20T18:04:02.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:04:02.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:04:02.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:04:02.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:04:02.273 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:02.271+0000 7fcd2b9b0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:02.275 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:02.273+0000 7fcd2b9b0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:02.275 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:02.274+0000 7fcd2b9b0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:02.354 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-20T18:04:02.354 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:04:02.354 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-20T18:04:02.354 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:04:02.355 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:04:02.355 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:04:02.355 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:04:02.355 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:04:02.356 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:04:02.356 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T18:04:02.457 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:04:02.506 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:02.503+0000 7fcd2b9b0900 -1 Falling back to public interface 2026-02-20T18:04:02.768 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:02.766+0000 7fcd2b9b0900 -1 osd.0 0 log_to_monitors true 2026-02-20T18:04:03.458 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:04:03.458 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:04:03.459 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:04:03.459 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:04:03.459 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:04:03.460 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T18:04:03.686 INFO:tasks.workunit.client.0.trial195.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/4000032879,v1:127.0.0.1:6803/4000032879] [v2:127.0.0.1:6804/4000032879,v1:127.0.0.1:6805/4000032879] exists,up c320cfec-a245-4205-990c-5665f9055a37 2026-02-20T18:04:03.690 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:04:03.690 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:04:03.690 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:04:03.690 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:04:03.690 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-20T18:04:03.690 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:04:03.690 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:04:03.690 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-20T18:04:03.690 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:04:03.691 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-20T18:04:03.691 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-20T18:04:03.691 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:04:03.691 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:04:03.691 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:04:03.691 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-20T18:04:03.691 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-20T18:04:03.691 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:04:03.691 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:04:03.691 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:04:03.691 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:04:03.691 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:04:03.692 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:04:03.692 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:04:03.692 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:04:03.692 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:04:03.692 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:04:03.692 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:04:03.692 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:04:03.692 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:04:03.692 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:04:03.692 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:04:03.692 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:04:03.692 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:04:03.693 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:04:03.693 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:04:03.693 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:04:03.693 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:04:03.693 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:04:03.693 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-20T18:04:03.693 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:04:03.693 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=423754ab-6e5d-43b1-92fb-eb2ddb5e5347 2026-02-20T18:04:03.693 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 423754ab-6e5d-43b1-92fb-eb2ddb5e5347' 2026-02-20T18:04:03.694 INFO:tasks.workunit.client.0.trial195.stdout:add osd1 423754ab-6e5d-43b1-92fb-eb2ddb5e5347 2026-02-20T18:04:03.694 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:04:03.704 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQATophpDdvpKRAA5W5uYPNaWSMJ0cJRgz9GOQ== 2026-02-20T18:04:03.704 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQATophpDdvpKRAA5W5uYPNaWSMJ0cJRgz9GOQ=="}' 2026-02-20T18:04:03.705 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 423754ab-6e5d-43b1-92fb-eb2ddb5e5347 -i td/test-erasure-eio/1/new.json 2026-02-20T18:04:03.936 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:04:03.944 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-20T18:04:03.944 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQATophpDdvpKRAA5W5uYPNaWSMJ0cJRgz9GOQ== --osd-uuid 423754ab-6e5d-43b1-92fb-eb2ddb5e5347 2026-02-20T18:04:03.966 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:03.964+0000 7f5c665cd900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:03.968 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:03.966+0000 7f5c665cd900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:03.969 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:03.967+0000 7f5c665cd900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:03.969 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:03.967+0000 7f5c665cd900 -1 bdev(0x564ae0c36c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-20T18:04:03.970 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:03.967+0000 7f5c665cd900 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-20T18:04:04.299 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-20T18:04:04.299 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:04:04.300 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-20T18:04:04.300 INFO:tasks.workunit.client.0.trial195.stdout:adding osd1 key to auth repository 2026-02-20T18:04:04.300 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:04:04.619 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-20T18:04:04.620 INFO:tasks.workunit.client.0.trial195.stdout:start osd.1 2026-02-20T18:04:04.621 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:04:04.621 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:04:04.621 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:04:04.621 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:04:04.640 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:04.638+0000 7fef4e546900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:04.641 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:04.640+0000 7fef4e546900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:04.642 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:04.641+0000 7fef4e546900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:04.789 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:04.788+0000 7fef4e546900 -1 Falling back to public interface 2026-02-20T18:04:04.847 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-20T18:04:04.847 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:04:04.847 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-20T18:04:04.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:04:04.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:04:04.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:04:04.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:04:04.848 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:04:04.849 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:04:04.849 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:04:05.078 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:04:05.149 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:05.147+0000 7fef4e546900 -1 osd.1 0 log_to_monitors true 2026-02-20T18:04:06.078 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:04:06.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:04:06.079 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:04:06.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:04:06.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:04:06.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:04:06.308 INFO:tasks.workunit.client.0.trial195.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1574981260,v1:127.0.0.1:6811/1574981260] [v2:127.0.0.1:6812/1574981260,v1:127.0.0.1:6813/1574981260] exists,up 423754ab-6e5d-43b1-92fb-eb2ddb5e5347 2026-02-20T18:04:06.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:04:06.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:04:06.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:04:06.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:04:06.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-20T18:04:06.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:04:06.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:04:06.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-20T18:04:06.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:04:06.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-20T18:04:06.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-20T18:04:06.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:04:06.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:04:06.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:04:06.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-20T18:04:06.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-20T18:04:06.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:04:06.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:04:06.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:04:06.310 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:04:06.310 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:04:06.310 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:04:06.310 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:04:06.310 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:04:06.310 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:04:06.310 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:04:06.310 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:04:06.310 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:04:06.310 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:04:06.310 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:04:06.311 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:04:06.311 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:04:06.311 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:04:06.311 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:04:06.311 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:04:06.311 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:04:06.311 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:04:06.311 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:04:06.311 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-20T18:04:06.311 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:04:06.312 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c732b8e9-b530-449e-b8c2-57b80a0444c1 2026-02-20T18:04:06.312 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 c732b8e9-b530-449e-b8c2-57b80a0444c1' 2026-02-20T18:04:06.312 INFO:tasks.workunit.client.0.trial195.stdout:add osd2 c732b8e9-b530-449e-b8c2-57b80a0444c1 2026-02-20T18:04:06.312 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:04:06.328 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAWophpa09+ExAAsi7LrsUop4YgwCa4iXasZQ== 2026-02-20T18:04:06.328 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAWophpa09+ExAAsi7LrsUop4YgwCa4iXasZQ=="}' 2026-02-20T18:04:06.328 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c732b8e9-b530-449e-b8c2-57b80a0444c1 -i td/test-erasure-eio/2/new.json 2026-02-20T18:04:06.567 INFO:tasks.workunit.client.0.trial195.stdout:2 2026-02-20T18:04:06.574 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-20T18:04:06.575 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAWophpa09+ExAAsi7LrsUop4YgwCa4iXasZQ== --osd-uuid c732b8e9-b530-449e-b8c2-57b80a0444c1 2026-02-20T18:04:06.597 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:06.595+0000 7f3936375900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:06.598 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:06.597+0000 7f3936375900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:06.599 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:06.598+0000 7f3936375900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:06.600 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:06.598+0000 7f3936375900 -1 bdev(0x561953932c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-20T18:04:06.600 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:06.598+0000 7f3936375900 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-20T18:04:06.914 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-20T18:04:06.914 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:04:06.914 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-20T18:04:06.914 INFO:tasks.workunit.client.0.trial195.stdout:adding osd2 key to auth repository 2026-02-20T18:04:06.915 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:04:07.234 INFO:tasks.workunit.client.0.trial195.stdout:start osd.2 2026-02-20T18:04:07.235 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-20T18:04:07.235 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:04:07.235 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:04:07.235 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:04:07.235 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:04:07.256 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:07.253+0000 7f4d83814900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:07.257 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:07.255+0000 7f4d83814900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:07.258 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:07.256+0000 7f4d83814900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:07.398 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:07.395+0000 7f4d83814900 -1 Falling back to public interface 2026-02-20T18:04:07.468 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-20T18:04:07.468 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:04:07.468 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-20T18:04:07.468 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:04:07.468 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:04:07.469 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:04:07.469 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:04:07.469 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:04:07.469 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:04:07.470 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:04:07.695 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:04:07.763 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:07.761+0000 7f4d83814900 -1 osd.2 0 log_to_monitors true 2026-02-20T18:04:08.696 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:04:08.697 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:04:08.697 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:04:08.698 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:04:08.698 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:04:08.698 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:04:08.927 INFO:tasks.workunit.client.0.trial195.stdout:osd.2 up in weight 1 up_from 19 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/6716225,v1:127.0.0.1:6819/6716225] [v2:127.0.0.1:6820/6716225,v1:127.0.0.1:6821/6716225] exists,up c732b8e9-b530-449e-b8c2-57b80a0444c1 2026-02-20T18:04:08.929 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:04:08.929 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:04:08.929 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:04:08.929 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:04:08.929 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-20T18:04:08.929 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:04:08.929 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:04:08.929 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-20T18:04:08.929 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:04:08.929 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-20T18:04:08.930 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-20T18:04:08.930 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:04:08.930 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:04:08.930 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:04:08.930 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-20T18:04:08.930 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-20T18:04:08.930 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:04:08.930 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:04:08.930 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:04:08.930 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:04:08.930 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:04:08.930 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:04:08.930 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:04:08.930 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:04:08.930 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:04:08.930 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:04:08.930 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:04:08.930 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:04:08.930 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:04:08.930 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:04:08.930 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:04:08.930 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:04:08.931 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:04:08.931 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:04:08.931 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:04:08.931 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:04:08.931 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:04:08.931 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:04:08.931 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-20T18:04:08.931 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:04:08.931 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4899aeaa-6410-47ca-9d41-deb4c80f6153 2026-02-20T18:04:08.931 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 4899aeaa-6410-47ca-9d41-deb4c80f6153' 2026-02-20T18:04:08.932 INFO:tasks.workunit.client.0.trial195.stdout:add osd3 4899aeaa-6410-47ca-9d41-deb4c80f6153 2026-02-20T18:04:08.932 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:04:08.946 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAYophpFqRTOBAAxwcH4rtP1ijTvnyV8jlskA== 2026-02-20T18:04:08.946 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAYophpFqRTOBAAxwcH4rtP1ijTvnyV8jlskA=="}' 2026-02-20T18:04:08.946 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4899aeaa-6410-47ca-9d41-deb4c80f6153 -i td/test-erasure-eio/3/new.json 2026-02-20T18:04:09.184 INFO:tasks.workunit.client.0.trial195.stdout:3 2026-02-20T18:04:09.192 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-20T18:04:09.194 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAYophpFqRTOBAAxwcH4rtP1ijTvnyV8jlskA== --osd-uuid 4899aeaa-6410-47ca-9d41-deb4c80f6153 2026-02-20T18:04:09.217 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:09.215+0000 7f3345219900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:09.218 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:09.216+0000 7f3345219900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:09.219 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:09.217+0000 7f3345219900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:09.220 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:09.218+0000 7f3345219900 -1 bdev(0x55b991c9ec00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-20T18:04:09.220 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:09.218+0000 7f3345219900 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-20T18:04:09.558 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-20T18:04:09.558 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:04:09.559 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-20T18:04:09.559 INFO:tasks.workunit.client.0.trial195.stdout:adding osd3 key to auth repository 2026-02-20T18:04:09.559 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:04:09.884 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-20T18:04:09.884 INFO:tasks.workunit.client.0.trial195.stdout:start osd.3 2026-02-20T18:04:09.884 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:04:09.884 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:04:09.884 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:04:09.884 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:04:09.905 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:09.903+0000 7f8ccd771900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:09.906 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:09.905+0000 7f8ccd771900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:09.907 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:09.906+0000 7f8ccd771900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:10.044 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:10.042+0000 7f8ccd771900 -1 Falling back to public interface 2026-02-20T18:04:10.116 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-20T18:04:10.116 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:04:10.116 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-20T18:04:10.116 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:04:10.116 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:04:10.116 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:04:10.116 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:04:10.116 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:04:10.116 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:04:10.116 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T18:04:10.346 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:04:10.375 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:10.373+0000 7f8ccd771900 -1 osd.3 0 log_to_monitors true 2026-02-20T18:04:11.348 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:04:11.348 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:04:11.348 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:04:11.348 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:04:11.348 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:04:11.348 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T18:04:11.580 INFO:tasks.workunit.client.0.trial195.stdout:osd.3 up in weight 1 up_from 26 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/91507923,v1:127.0.0.1:6827/91507923] [v2:127.0.0.1:6828/91507923,v1:127.0.0.1:6829/91507923] exists,up 4899aeaa-6410-47ca-9d41-deb4c80f6153 2026-02-20T18:04:11.582 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:04:11.582 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:04:11.582 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:04:11.582 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:04:11.582 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-20T18:04:11.582 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:04:11.582 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:04:11.582 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-20T18:04:11.582 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:04:11.582 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-20T18:04:11.582 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-20T18:04:11.582 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:04:11.583 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:04:11.583 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:04:11.583 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-20T18:04:11.583 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-20T18:04:11.583 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:04:11.583 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:04:11.583 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:04:11.583 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:04:11.583 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:04:11.583 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:04:11.583 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:04:11.583 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:04:11.583 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:04:11.583 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:04:11.583 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:04:11.583 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:04:11.583 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:04:11.583 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:04:11.583 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:04:11.583 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:04:11.583 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:04:11.583 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:04:11.583 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:04:11.584 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:04:11.584 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:04:11.584 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:04:11.584 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-20T18:04:11.584 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:04:11.584 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ab695fc3-1d0f-491c-bd19-2f1bfc06e20c 2026-02-20T18:04:11.584 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 ab695fc3-1d0f-491c-bd19-2f1bfc06e20c' 2026-02-20T18:04:11.584 INFO:tasks.workunit.client.0.trial195.stdout:add osd4 ab695fc3-1d0f-491c-bd19-2f1bfc06e20c 2026-02-20T18:04:11.584 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:04:11.600 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAbophpJQa3IxAAe1zHGni+zFnI9djuhYQICw== 2026-02-20T18:04:11.600 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAbophpJQa3IxAAe1zHGni+zFnI9djuhYQICw=="}' 2026-02-20T18:04:11.600 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ab695fc3-1d0f-491c-bd19-2f1bfc06e20c -i td/test-erasure-eio/4/new.json 2026-02-20T18:04:11.836 INFO:tasks.workunit.client.0.trial195.stdout:4 2026-02-20T18:04:11.843 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-20T18:04:11.844 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAbophpJQa3IxAAe1zHGni+zFnI9djuhYQICw== --osd-uuid ab695fc3-1d0f-491c-bd19-2f1bfc06e20c 2026-02-20T18:04:11.867 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:11.865+0000 7f61c5a10900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:11.868 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:11.867+0000 7f61c5a10900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:11.869 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:11.868+0000 7f61c5a10900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:11.870 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:11.868+0000 7f61c5a10900 -1 bdev(0x5586b378ac00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-20T18:04:11.870 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:11.868+0000 7f61c5a10900 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-20T18:04:12.199 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-20T18:04:12.199 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:04:12.200 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-20T18:04:12.200 INFO:tasks.workunit.client.0.trial195.stdout:adding osd4 key to auth repository 2026-02-20T18:04:12.200 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:04:12.517 INFO:tasks.workunit.client.0.trial195.stdout:start osd.4 2026-02-20T18:04:12.518 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-20T18:04:12.518 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:04:12.518 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:04:12.518 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:04:12.518 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:04:12.539 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:12.537+0000 7f31f5e0f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:12.540 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:12.538+0000 7f31f5e0f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:12.541 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:12.539+0000 7f31f5e0f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:12.677 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:12.674+0000 7f31f5e0f900 -1 Falling back to public interface 2026-02-20T18:04:12.753 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-20T18:04:12.753 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:04:12.753 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-20T18:04:12.753 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:04:12.753 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:04:12.753 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:04:12.754 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:04:12.754 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:04:12.754 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:04:12.754 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-20T18:04:12.982 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:12.980+0000 7f31f5e0f900 -1 osd.4 0 log_to_monitors true 2026-02-20T18:04:12.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:04:13.986 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:04:13.987 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:04:13.987 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:04:13.988 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:04:13.988 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:04:13.988 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-20T18:04:14.222 INFO:tasks.workunit.client.0.trial195.stdout:osd.4 up in weight 1 up_from 36 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/1570530269,v1:127.0.0.1:6835/1570530269] [v2:127.0.0.1:6836/1570530269,v1:127.0.0.1:6837/1570530269] exists,up ab695fc3-1d0f-491c-bd19-2f1bfc06e20c 2026-02-20T18:04:14.224 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:04:14.224 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:04:14.224 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:04:14.224 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-20T18:04:14.225 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-20T18:04:14.225 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-20T18:04:14.225 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:04:14.225 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:04:14.225 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:04:14.225 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.0.asok 2026-02-20T18:04:14.225 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-20T18:04:14.225 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-osd.0.asok log flush 2026-02-20T18:04:14.270 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T18:04:14.273 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-20T18:04:14.276 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T18:04:02.506+0000 7fcd2b9b0900 0 load: jerasure load: lrc 2026-02-20T18:04:14.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: CEPH_ARGS='--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:04:14.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:641: TEST_ec_recovery_unfound: local poolname=pool-jerasure 2026-02-20T18:04:14.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:642: TEST_ec_recovery_unfound: create_erasure_coded_pool pool-jerasure 3 2 2026-02-20T18:04:14.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-20T18:04:14.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-20T18:04:14.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-20T18:04:14.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-20T18:04:14.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-20T18:04:14.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-20T18:04:14.278 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-20T18:04:14.611 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-20T18:04:14.612 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-20T18:04:14.983 INFO:tasks.workunit.client.0.trial195.stderr:pool 'pool-jerasure' already exists 2026-02-20T18:04:14.991 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-20T18:04:15.993 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-20T18:04:15.993 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:04:15.993 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:04:15.993 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:04:15.993 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:04:15.994 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:04:15.994 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:04:15.995 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:04:15.995 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:04:15.995 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:04:16.080 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:04:16.080 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:04:16.080 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:04:16.080 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:04:16.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:04:16.081 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:04:16.303 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:04:16.304 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:04:16.304 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:04:16.304 INFO:tasks.workunit.client.0.trial195.stderr:3 2026-02-20T18:04:16.304 INFO:tasks.workunit.client.0.trial195.stderr:4' 2026-02-20T18:04:16.304 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:04:16.304 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:04:16.305 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:04:16.370 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803781 2026-02-20T18:04:16.371 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803781 2026-02-20T18:04:16.371 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781' 2026-02-20T18:04:16.371 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:04:16.371 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:04:16.436 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574853 2026-02-20T18:04:16.436 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574853 2026-02-20T18:04:16.436 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781 1-55834574853' 2026-02-20T18:04:16.436 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:04:16.436 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:04:16.504 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378628 2026-02-20T18:04:16.504 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378628 2026-02-20T18:04:16.504 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781 1-55834574853 2-81604378628' 2026-02-20T18:04:16.504 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:04:16.504 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:04:16.568 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149700 2026-02-20T18:04:16.568 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149700 2026-02-20T18:04:16.568 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781 1-55834574853 2-81604378628 3-111669149700' 2026-02-20T18:04:16.568 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:04:16.569 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-20T18:04:16.632 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=154618822659 2026-02-20T18:04:16.632 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 154618822659 2026-02-20T18:04:16.632 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781 1-55834574853 2-81604378628 3-111669149700 4-154618822659' 2026-02-20T18:04:16.633 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:04:16.633 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803781 2026-02-20T18:04:16.633 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:04:16.633 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:04:16.634 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803781 2026-02-20T18:04:16.634 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:04:16.635 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803781 2026-02-20T18:04:16.635 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.0 seq 25769803781 2026-02-20T18:04:16.635 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-20T18:04:16.636 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:04:16.864 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803781 -lt 25769803781 2026-02-20T18:04:16.864 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:04:16.865 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574853 2026-02-20T18:04:16.865 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:04:16.865 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:04:16.866 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-55834574853 2026-02-20T18:04:16.866 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:04:16.867 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574853 2026-02-20T18:04:16.867 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 55834574853' 2026-02-20T18:04:16.867 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.1 seq 55834574853 2026-02-20T18:04:16.867 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:04:17.094 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574851 -lt 55834574853 2026-02-20T18:04:17.094 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:04:18.095 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:04:18.095 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:04:18.319 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574851 -lt 55834574853 2026-02-20T18:04:18.319 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:04:19.320 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-20T18:04:19.320 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:04:19.543 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574853 -lt 55834574853 2026-02-20T18:04:19.544 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:04:19.544 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-81604378628 2026-02-20T18:04:19.544 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:04:19.544 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:04:19.545 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-81604378628 2026-02-20T18:04:19.545 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:04:19.546 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=81604378628 2026-02-20T18:04:19.546 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 81604378628' 2026-02-20T18:04:19.547 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.2 seq 81604378628 2026-02-20T18:04:19.547 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:04:19.768 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 81604378628 -lt 81604378628 2026-02-20T18:04:19.768 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:04:19.769 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149700 2026-02-20T18:04:19.769 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:04:19.770 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:04:19.770 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149700 2026-02-20T18:04:19.770 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:04:19.772 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149700 2026-02-20T18:04:19.772 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149700' 2026-02-20T18:04:19.772 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.3 seq 111669149700 2026-02-20T18:04:19.772 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:04:19.995 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149700 -lt 111669149700 2026-02-20T18:04:19.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:04:19.996 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-154618822659 2026-02-20T18:04:19.997 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:04:19.997 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-20T18:04:19.998 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-154618822659 2026-02-20T18:04:19.998 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:04:19.999 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=154618822659 2026-02-20T18:04:19.999 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 154618822659' 2026-02-20T18:04:19.999 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.4 seq 154618822659 2026-02-20T18:04:20.000 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-20T18:04:20.226 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 154618822659 -lt 154618822659 2026-02-20T18:04:20.226 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:04:20.227 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:04:20.227 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:04:20.541 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:04:20.541 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:04:20.541 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:04:20.541 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:04:20.541 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:04:20.541 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:04:20.541 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:04:20.542 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:04:20.764 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:04:20.765 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:04:20.765 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:04:20.766 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:04:21.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:04:21.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:04:21.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:04:21.082 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:644: TEST_ec_recovery_unfound: ceph pg dump pgs 2026-02-20T18:04:21.294 INFO:tasks.workunit.client.0.trial195.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-20T18:04:21.294 INFO:tasks.workunit.client.0.trial195.stdout:2.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-20T18:04:15.472059+0000 0'0 44:18 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-20T18:04:14.921735+0000 0'0 2026-02-20T18:04:14.921735+0000 0 0 periodic scrub scheduled @ 2026-02-22T03:31:26.623200+0000 0 0 2026-02-20T18:04:21.295 INFO:tasks.workunit.client.0.trial195.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-20T18:04:15.678169+0000 0'0 44:15 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-20T18:04:00.492751+0000 0'0 2026-02-20T18:04:00.492751+0000 0 0 periodic scrub scheduled @ 2026-02-21T20:16:38.930900+0000 0 0 2026-02-20T18:04:21.295 INFO:tasks.workunit.client.0.trial195.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-20T18:04:15.026075+0000 0'0 44:105 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-20T18:04:00.492751+0000 0'0 2026-02-20T18:04:00.492751+0000 0 0 periodic scrub scheduled @ 2026-02-21T19:56:50.166315+0000 0 0 2026-02-20T18:04:21.295 INFO:tasks.workunit.client.0.trial195.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-20T18:04:15.471294+0000 0'0 44:15 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-20T18:04:00.492751+0000 0'0 2026-02-20T18:04:00.492751+0000 0 0 periodic scrub scheduled @ 2026-02-22T05:56:58.141451+0000 0 0 2026-02-20T18:04:21.295 INFO:tasks.workunit.client.0.trial195.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-20T18:04:08.507712+0000 0'0 44:68 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-20T18:04:00.492751+0000 0'0 2026-02-20T18:04:00.492751+0000 0 0 periodic scrub scheduled @ 2026-02-21T18:11:30.660658+0000 0 0 2026-02-20T18:04:21.295 INFO:tasks.workunit.client.0.trial195.stdout: 2026-02-20T18:04:21.295 INFO:tasks.workunit.client.0.trial195.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-20T18:04:21.295 INFO:tasks.workunit.client.0.trial195.stderr:dumped pgs 2026-02-20T18:04:21.302 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:646: TEST_ec_recovery_unfound: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-20T18:04:21.302 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-20T18:04:21.303 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-20T18:04:21.303 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-20T18:04:21.303 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:04:21.303 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-20T18:04:21.303 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:04:21.303 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-20T18:04:21.303 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:04:21.303 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-20T18:04:21.303 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:04:21.303 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-20T18:04:21.303 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:21.322 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:648: TEST_ec_recovery_unfound: get_osds pool-jerasure myobject 2026-02-20T18:04:21.322 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:04:21.323 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-20T18:04:21.323 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-20T18:04:21.323 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:04:21.554 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T18:04:21.554 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:04:21.554 INFO:tasks.workunit.client.0.trial195.stderr:4 2026-02-20T18:04:21.555 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T18:04:21.555 INFO:tasks.workunit.client.0.trial195.stderr:2' 2026-02-20T18:04:21.555 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2 2026-02-20T18:04:21.555 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:648: TEST_ec_recovery_unfound: initial_osds=('3' '1' '4' '0' '2') 2026-02-20T18:04:21.555 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:648: TEST_ec_recovery_unfound: local -a initial_osds 2026-02-20T18:04:21.555 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:649: TEST_ec_recovery_unfound: local last_osd=2 2026-02-20T18:04:21.555 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:650: TEST_ec_recovery_unfound: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-20T18:04:21.555 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:04:21.555 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:04:21.555 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:04:21.555 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:04:21.555 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:04:21.659 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:04:21.659 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:651: TEST_ec_recovery_unfound: ceph osd down 2 2026-02-20T18:04:21.885 INFO:tasks.workunit.client.0.trial195.stderr:osd.2 is already down. 2026-02-20T18:04:21.893 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:652: TEST_ec_recovery_unfound: ceph osd out 2 2026-02-20T18:04:22.139 INFO:tasks.workunit.client.0.trial195.stderr:osd.2 is already out. 2026-02-20T18:04:22.147 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:654: TEST_ec_recovery_unfound: ceph pg dump pgs 2026-02-20T18:04:22.365 INFO:tasks.workunit.client.0.trial195.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-20T18:04:22.365 INFO:tasks.workunit.client.0.trial195.stdout:2.0 1 0 0 0 0 4096 0 0 1 0 1 active+clean 2026-02-20T18:04:15.472059+0000 44'1 44:19 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-20T18:04:14.921735+0000 0'0 2026-02-20T18:04:14.921735+0000 0 0 periodic scrub scheduled @ 2026-02-22T03:31:26.623200+0000 0 0 2026-02-20T18:04:22.365 INFO:tasks.workunit.client.0.trial195.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-20T18:04:15.678169+0000 0'0 44:15 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-20T18:04:00.492751+0000 0'0 2026-02-20T18:04:00.492751+0000 0 0 periodic scrub scheduled @ 2026-02-21T20:16:38.930900+0000 0 0 2026-02-20T18:04:22.365 INFO:tasks.workunit.client.0.trial195.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-20T18:04:15.026075+0000 0'0 44:105 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-20T18:04:00.492751+0000 0'0 2026-02-20T18:04:00.492751+0000 0 0 periodic scrub scheduled @ 2026-02-21T19:56:50.166315+0000 0 0 2026-02-20T18:04:22.366 INFO:tasks.workunit.client.0.trial195.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-20T18:04:15.471294+0000 0'0 44:15 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-20T18:04:00.492751+0000 0'0 2026-02-20T18:04:00.492751+0000 0 0 periodic scrub scheduled @ 2026-02-22T05:56:58.141451+0000 0 0 2026-02-20T18:04:22.366 INFO:tasks.workunit.client.0.trial195.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-20T18:04:08.507712+0000 0'0 44:68 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-20T18:04:00.492751+0000 0'0 2026-02-20T18:04:00.492751+0000 0 0 periodic scrub scheduled @ 2026-02-21T18:11:30.660658+0000 0 0 2026-02-20T18:04:22.366 INFO:tasks.workunit.client.0.trial195.stdout: 2026-02-20T18:04:22.366 INFO:tasks.workunit.client.0.trial195.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-20T18:04:22.366 INFO:tasks.workunit.client.0.trial195.stderr:dumped pgs 2026-02-20T18:04:22.372 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:656: TEST_ec_recovery_unfound: dd if=/dev/urandom of=td/test-erasure-eio/ORIGINAL bs=1024 count=4 2026-02-20T18:04:22.373 INFO:tasks.workunit.client.0.trial195.stderr:4+0 records in 2026-02-20T18:04:22.373 INFO:tasks.workunit.client.0.trial195.stderr:4+0 records out 2026-02-20T18:04:22.373 INFO:tasks.workunit.client.0.trial195.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 7.3827e-05 s, 55.5 MB/s 2026-02-20T18:04:22.374 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: seq 1 100 2026-02-20T18:04:22.374 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:22.375 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj1 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:22.396 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:22.396 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj2 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:22.416 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:22.416 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj3 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:22.434 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:22.434 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj4 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:22.455 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:22.455 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj5 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:22.475 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:22.475 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj6 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:22.493 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:22.493 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj7 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:22.512 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:22.512 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj8 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:22.531 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:22.531 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj9 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:22.550 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:22.550 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj10 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:22.569 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:22.569 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj11 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:22.588 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:22.588 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj12 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:22.607 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:22.607 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj13 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:22.626 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:22.626 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj14 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:22.646 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:22.646 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj15 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:22.666 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:22.666 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj16 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:22.684 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:22.684 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj17 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:22.703 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:22.703 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj18 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:22.722 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:22.722 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj19 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:22.742 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:22.742 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj20 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:22.761 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:22.761 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj21 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:22.780 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:22.780 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj22 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:22.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:22.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj23 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:22.817 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:22.817 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj24 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:22.835 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:22.835 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj25 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:22.854 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:22.854 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj26 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:22.874 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:22.874 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj27 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:22.893 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:22.893 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj28 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:22.912 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:22.912 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj29 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:22.931 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:22.931 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj30 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:22.950 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:22.950 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj31 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:22.969 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:22.969 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj32 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:22.987 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:22.988 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj33 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.006 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.006 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj34 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.026 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.026 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj35 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.044 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.044 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj36 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.064 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.064 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj37 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.083 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.083 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj38 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.102 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.102 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj39 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.121 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.121 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj40 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.139 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.140 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj41 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.158 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.159 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj42 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.178 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.178 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj43 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.197 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.197 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj44 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.216 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.216 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj45 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.235 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.235 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj46 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.253 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.253 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj47 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.272 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.272 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj48 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.291 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.291 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj49 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.310 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.310 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj50 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.329 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.330 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj51 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.349 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.349 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj52 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.368 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.368 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj53 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.387 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.387 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj54 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.406 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.406 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj55 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.426 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.426 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj56 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.444 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.444 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj57 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.463 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.463 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj58 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.481 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.481 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj59 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.500 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.500 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj60 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.518 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.518 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj61 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.537 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.537 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj62 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.556 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.557 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj63 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.574 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.574 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj64 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.593 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.594 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj65 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.613 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.613 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj66 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.633 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.633 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj67 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.652 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.652 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj68 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.672 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.672 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj69 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.691 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.691 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj70 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.710 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.711 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj71 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.729 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.729 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj72 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.749 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.749 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj73 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.767 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.767 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj74 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.786 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.786 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj75 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.806 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.806 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj76 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.825 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.825 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj77 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.845 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.845 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj78 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.864 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.864 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj79 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.883 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.883 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj80 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.901 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.902 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj81 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.920 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.920 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj82 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.940 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.940 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj83 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.958 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.958 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj84 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.977 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.977 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj85 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:23.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:23.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj86 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:24.014 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:24.014 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj87 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:24.034 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:24.034 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj88 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:24.052 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:24.053 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj89 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:24.072 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:24.072 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj90 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:24.092 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:24.092 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj91 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:24.111 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:24.111 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj92 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:24.130 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:24.130 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj93 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:24.150 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:24.150 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj94 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:24.169 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:24.169 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj95 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:24.187 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:24.187 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj96 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:24.206 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:24.206 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj97 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:24.224 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:24.225 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj98 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:24.243 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:24.243 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj99 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:24.262 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:24.262 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj100 td/test-erasure-eio/ORIGINAL 2026-02-20T18:04:24.284 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:662: TEST_ec_recovery_unfound: inject_eio ec data pool-jerasure obj75 td/test-erasure-eio 0 2026-02-20T18:04:24.284 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=ec 2026-02-20T18:04:24.284 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-20T18:04:24.284 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=data 2026-02-20T18:04:24.284 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-20T18:04:24.285 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=pool-jerasure 2026-02-20T18:04:24.285 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-20T18:04:24.285 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=obj75 2026-02-20T18:04:24.285 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-20T18:04:24.285 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/test-erasure-eio 2026-02-20T18:04:24.285 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-20T18:04:24.285 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=0 2026-02-20T18:04:24.285 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-20T18:04:24.285 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds pool-jerasure obj75 2026-02-20T18:04:24.286 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:04:24.286 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj75 2026-02-20T18:04:24.286 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj75 2026-02-20T18:04:24.286 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:04:24.508 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T18:04:24.508 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:04:24.508 INFO:tasks.workunit.client.0.trial195.stderr:4 2026-02-20T18:04:24.508 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T18:04:24.508 INFO:tasks.workunit.client.0.trial195.stderr:2147483647' 2026-02-20T18:04:24.509 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2147483647 2026-02-20T18:04:24.509 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('3' '1' '4' '0' '2147483647') 2026-02-20T18:04:24.509 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-20T18:04:24.509 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=3 2026-02-20T18:04:24.509 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' ec '!=' ec ']' 2026-02-20T18:04:24.509 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/test-erasure-eio/3/type 2026-02-20T18:04:24.510 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-20T18:04:24.510 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-20T18:04:24.510 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-20T18:04:24.510 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-20T18:04:24.510 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-20T18:04:24.510 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-20T18:04:24.510 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-20T18:04:24.510 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-20T18:04:24.510 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-20T18:04:24.511 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-20T18:04:24.511 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:04:24.511 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:04:24.511 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:04:24.511 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.3.asok 2026-02-20T18:04:24.511 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.85808/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-20T18:04:24.559 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-20T18:04:24.559 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-20T18:04:24.560 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-20T18:04:24.560 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.3 2026-02-20T18:04:24.560 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-20T18:04:24.561 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-20T18:04:24.561 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:04:24.561 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:04:24.561 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:04:24.561 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.3.asok 2026-02-20T18:04:24.562 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-20T18:04:24.562 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-osd.3.asok injectdataerr pool-jerasure obj75 0 2026-02-20T18:04:24.612 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:663: TEST_ec_recovery_unfound: inject_eio ec data pool-jerasure obj75 td/test-erasure-eio 1 2026-02-20T18:04:24.612 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=ec 2026-02-20T18:04:24.612 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-20T18:04:24.612 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=data 2026-02-20T18:04:24.612 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-20T18:04:24.612 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=pool-jerasure 2026-02-20T18:04:24.612 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-20T18:04:24.612 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=obj75 2026-02-20T18:04:24.612 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-20T18:04:24.613 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/test-erasure-eio 2026-02-20T18:04:24.613 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-20T18:04:24.613 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=1 2026-02-20T18:04:24.613 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-20T18:04:24.613 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds pool-jerasure obj75 2026-02-20T18:04:24.613 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:04:24.613 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj75 2026-02-20T18:04:24.613 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj75 2026-02-20T18:04:24.613 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:04:24.839 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T18:04:24.840 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:04:24.840 INFO:tasks.workunit.client.0.trial195.stderr:4 2026-02-20T18:04:24.840 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T18:04:24.840 INFO:tasks.workunit.client.0.trial195.stderr:2147483647' 2026-02-20T18:04:24.840 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2147483647 2026-02-20T18:04:24.840 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('3' '1' '4' '0' '2147483647') 2026-02-20T18:04:24.840 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-20T18:04:24.840 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=1 2026-02-20T18:04:24.840 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' ec '!=' ec ']' 2026-02-20T18:04:24.840 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/test-erasure-eio/1/type 2026-02-20T18:04:24.842 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-20T18:04:24.842 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-20T18:04:24.842 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-20T18:04:24.843 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-20T18:04:24.843 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-20T18:04:24.843 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-20T18:04:24.843 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-20T18:04:24.843 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-20T18:04:24.843 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-20T18:04:24.843 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-20T18:04:24.843 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:04:24.843 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:04:24.843 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:04:24.843 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.1.asok 2026-02-20T18:04:24.843 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.85808/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-20T18:04:24.894 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-20T18:04:24.895 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-20T18:04:24.896 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-20T18:04:24.896 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.1 2026-02-20T18:04:24.896 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-20T18:04:24.896 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-20T18:04:24.896 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:04:24.896 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:04:24.896 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:04:24.896 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.1.asok 2026-02-20T18:04:24.896 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-20T18:04:24.896 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-osd.1.asok injectdataerr pool-jerasure obj75 1 2026-02-20T18:04:24.947 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:665: TEST_ec_recovery_unfound: activate_osd td/test-erasure-eio 2 2026-02-20T18:04:24.947 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-20T18:04:24.948 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-20T18:04:24.948 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-20T18:04:24.948 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-20T18:04:24.948 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-20T18:04:24.948 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:04:24.948 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:04:24.948 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:04:24.948 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:04:24.948 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-20T18:04:24.948 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-20T18:04:24.949 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-20T18:04:24.949 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-20T18:04:24.949 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:04:24.949 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-20T18:04:24.949 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:04:24.949 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:04:24.949 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:04:24.949 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:04:24.949 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:04:24.949 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:04:24.949 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:04:24.950 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:04:24.950 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:04:24.950 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:04:24.950 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:04:24.950 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:04:24.950 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:04:24.950 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:04:24.950 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-20T18:04:24.950 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-20T18:04:24.950 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-20T18:04:24.950 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-20T18:04:24.951 INFO:tasks.workunit.client.0.trial195.stdout:start osd.2 2026-02-20T18:04:24.952 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:04:24.952 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-20T18:04:24.952 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-20T18:04:24.952 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-20T18:04:24.954 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-20T18:04:24.954 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-20T18:04:24.964 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:24.962+0000 7ff536dcd900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:24.965 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:24.963+0000 7ff536dcd900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:24.965 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:24.964+0000 7ff536dcd900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:25.094 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:25.092+0000 7ff536dcd900 -1 Falling back to public interface 2026-02-20T18:04:25.172 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-20T18:04:25.172 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:04:25.172 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-20T18:04:25.172 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:04:25.172 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:04:25.172 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:04:25.172 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:04:25.173 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:04:25.173 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:04:25.174 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:04:25.403 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:04:25.407 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:25.406+0000 7ff536dcd900 -1 osd.2 44 log_to_monitors true 2026-02-20T18:04:26.404 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:04:26.404 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:04:26.404 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:04:26.404 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:04:26.404 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:04:26.404 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:04:26.634 INFO:tasks.workunit.client.0.trial195.stdout:osd.2 up out weight 0 up_from 50 up_thru 19 down_at 45 last_clean_interval [19,44) [v2:127.0.0.1:6818/971055349,v1:127.0.0.1:6819/971055349] [v2:127.0.0.1:6820/971055349,v1:127.0.0.1:6821/971055349] exists,up c732b8e9-b530-449e-b8c2-57b80a0444c1 2026-02-20T18:04:26.635 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:04:26.635 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:04:26.635 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:04:26.635 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:666: TEST_ec_recovery_unfound: ceph osd in 2 2026-02-20T18:04:26.911 INFO:tasks.workunit.client.0.trial195.stderr:osd.2 is already in. 2026-02-20T18:04:26.919 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:668: TEST_ec_recovery_unfound: sleep 15 2026-02-20T18:04:41.921 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:670: TEST_ec_recovery_unfound: seq 1 100 2026-02-20T18:04:41.922 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:670: TEST_ec_recovery_unfound: for tmp in $(seq 1 100) 2026-02-20T18:04:41.922 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:671: TEST_ec_recovery_unfound: get_state 2.0 2026-02-20T18:04:41.923 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:60: get_state: local pgid=2.0 2026-02-20T18:04:41.923 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:61: get_state: local sname=state 2026-02-20T18:04:41.923 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:62: get_state: ceph --format json pg dump pgs 2026-02-20T18:04:41.923 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:63: get_state: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .state' 2026-02-20T18:04:42.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:671: TEST_ec_recovery_unfound: state=active+recovery_unfound+degraded 2026-02-20T18:04:42.146 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:672: TEST_ec_recovery_unfound: echo active+recovery_unfound+degraded 2026-02-20T18:04:42.146 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:672: TEST_ec_recovery_unfound: grep recovery_unfound 2026-02-20T18:04:42.146 INFO:tasks.workunit.client.0.trial195.stdout:active+recovery_unfound+degraded 2026-02-20T18:04:42.147 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:673: TEST_ec_recovery_unfound: '[' 0 = 0 ']' 2026-02-20T18:04:42.147 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:674: TEST_ec_recovery_unfound: break 2026-02-20T18:04:42.147 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: ceph pg dump pgs 2026-02-20T18:04:42.361 INFO:tasks.workunit.client.0.trial195.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-20T18:04:42.361 INFO:tasks.workunit.client.0.trial195.stdout:2.0 101 1 3 0 1 413696 0 0 101 0 101 active+recovery_unfound+degraded 2026-02-20T18:04:26.952389+0000 48'101 55:364 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-20T18:04:14.921735+0000 0'0 2026-02-20T18:04:14.921735+0000 0 0 no scrub is scheduled 0 0 2026-02-20T18:04:42.361 INFO:tasks.workunit.client.0.trial195.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-20T18:04:26.950891+0000 0'0 55:55 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-20T18:04:00.492751+0000 0'0 2026-02-20T18:04:00.492751+0000 0 0 periodic scrub scheduled @ 2026-02-22T01:28:27.979460+0000 0 0 2026-02-20T18:04:42.361 INFO:tasks.workunit.client.0.trial195.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-20T18:04:15.026075+0000 0'0 55:125 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-20T18:04:00.492751+0000 0'0 2026-02-20T18:04:00.492751+0000 0 0 periodic scrub scheduled @ 2026-02-21T19:56:50.166315+0000 0 0 2026-02-20T18:04:42.361 INFO:tasks.workunit.client.0.trial195.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-20T18:04:15.471294+0000 0'0 55:37 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-20T18:04:00.492751+0000 0'0 2026-02-20T18:04:00.492751+0000 0 0 periodic scrub scheduled @ 2026-02-22T05:56:58.141451+0000 0 0 2026-02-20T18:04:42.361 INFO:tasks.workunit.client.0.trial195.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-20T18:04:26.950895+0000 0'0 55:107 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-20T18:04:00.492751+0000 0'0 2026-02-20T18:04:00.492751+0000 0 0 periodic scrub scheduled @ 2026-02-22T00:26:05.020359+0000 0 0 2026-02-20T18:04:42.361 INFO:tasks.workunit.client.0.trial195.stdout: 2026-02-20T18:04:42.361 INFO:tasks.workunit.client.0.trial195.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-20T18:04:42.362 INFO:tasks.workunit.client.0.trial195.stderr:dumped pgs 2026-02-20T18:04:42.368 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:681: TEST_ec_recovery_unfound: ceph pg 2.0 list_unfound 2026-02-20T18:04:42.430 INFO:tasks.workunit.client.0.trial195.stdout:{ 2026-02-20T18:04:42.430 INFO:tasks.workunit.client.0.trial195.stdout: "num_missing": 1, 2026-02-20T18:04:42.430 INFO:tasks.workunit.client.0.trial195.stdout: "num_unfound": 1, 2026-02-20T18:04:42.430 INFO:tasks.workunit.client.0.trial195.stdout: "objects": [ 2026-02-20T18:04:42.430 INFO:tasks.workunit.client.0.trial195.stdout: { 2026-02-20T18:04:42.430 INFO:tasks.workunit.client.0.trial195.stdout: "oid": { 2026-02-20T18:04:42.430 INFO:tasks.workunit.client.0.trial195.stdout: "oid": "obj75", 2026-02-20T18:04:42.430 INFO:tasks.workunit.client.0.trial195.stdout: "key": "", 2026-02-20T18:04:42.430 INFO:tasks.workunit.client.0.trial195.stdout: "snapid": -2, 2026-02-20T18:04:42.430 INFO:tasks.workunit.client.0.trial195.stdout: "hash": 3565005477, 2026-02-20T18:04:42.430 INFO:tasks.workunit.client.0.trial195.stdout: "max": 0, 2026-02-20T18:04:42.431 INFO:tasks.workunit.client.0.trial195.stdout: "pool": 2, 2026-02-20T18:04:42.431 INFO:tasks.workunit.client.0.trial195.stdout: "namespace": "" 2026-02-20T18:04:42.431 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:04:42.431 INFO:tasks.workunit.client.0.trial195.stdout: "need": "48'76", 2026-02-20T18:04:42.431 INFO:tasks.workunit.client.0.trial195.stdout: "have": "0'0", 2026-02-20T18:04:42.431 INFO:tasks.workunit.client.0.trial195.stdout: "flags": "none", 2026-02-20T18:04:42.431 INFO:tasks.workunit.client.0.trial195.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2026-02-20T18:04:42.431 INFO:tasks.workunit.client.0.trial195.stdout: "locations": [ 2026-02-20T18:04:42.431 INFO:tasks.workunit.client.0.trial195.stdout: "0(3)", 2026-02-20T18:04:42.431 INFO:tasks.workunit.client.0.trial195.stdout: "4(2)" 2026-02-20T18:04:42.431 INFO:tasks.workunit.client.0.trial195.stdout: ] 2026-02-20T18:04:42.431 INFO:tasks.workunit.client.0.trial195.stdout: } 2026-02-20T18:04:42.431 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:04:42.431 INFO:tasks.workunit.client.0.trial195.stdout: "state": "NotRecovering", 2026-02-20T18:04:42.431 INFO:tasks.workunit.client.0.trial195.stdout: "available_might_have_unfound": true, 2026-02-20T18:04:42.431 INFO:tasks.workunit.client.0.trial195.stdout: "might_have_unfound": [], 2026-02-20T18:04:42.431 INFO:tasks.workunit.client.0.trial195.stdout: "more": false 2026-02-20T18:04:42.431 INFO:tasks.workunit.client.0.trial195.stdout:} 2026-02-20T18:04:42.435 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: ceph pg 2.0 query 2026-02-20T18:04:42.499 INFO:tasks.workunit.client.0.trial195.stdout:{ 2026-02-20T18:04:42.500 INFO:tasks.workunit.client.0.trial195.stdout: "snap_trimq": "[]", 2026-02-20T18:04:42.500 INFO:tasks.workunit.client.0.trial195.stdout: "snap_trimq_len": 0, 2026-02-20T18:04:42.500 INFO:tasks.workunit.client.0.trial195.stdout: "state": "active+recovery_unfound+degraded", 2026-02-20T18:04:42.500 INFO:tasks.workunit.client.0.trial195.stdout: "epoch": 55, 2026-02-20T18:04:42.500 INFO:tasks.workunit.client.0.trial195.stdout: "up": [ 2026-02-20T18:04:42.500 INFO:tasks.workunit.client.0.trial195.stdout: 3, 2026-02-20T18:04:42.500 INFO:tasks.workunit.client.0.trial195.stdout: 1, 2026-02-20T18:04:42.500 INFO:tasks.workunit.client.0.trial195.stdout: 4, 2026-02-20T18:04:42.500 INFO:tasks.workunit.client.0.trial195.stdout: 0, 2026-02-20T18:04:42.501 INFO:tasks.workunit.client.0.trial195.stdout: 2 2026-02-20T18:04:42.501 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:04:42.501 INFO:tasks.workunit.client.0.trial195.stdout: "acting": [ 2026-02-20T18:04:42.501 INFO:tasks.workunit.client.0.trial195.stdout: 3, 2026-02-20T18:04:42.501 INFO:tasks.workunit.client.0.trial195.stdout: 1, 2026-02-20T18:04:42.501 INFO:tasks.workunit.client.0.trial195.stdout: 4, 2026-02-20T18:04:42.501 INFO:tasks.workunit.client.0.trial195.stdout: 0, 2026-02-20T18:04:42.501 INFO:tasks.workunit.client.0.trial195.stdout: 2 2026-02-20T18:04:42.501 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:04:42.501 INFO:tasks.workunit.client.0.trial195.stdout: "acting_recovery_backfill": [ 2026-02-20T18:04:42.501 INFO:tasks.workunit.client.0.trial195.stdout: "0(3)", 2026-02-20T18:04:42.501 INFO:tasks.workunit.client.0.trial195.stdout: "1(1)", 2026-02-20T18:04:42.502 INFO:tasks.workunit.client.0.trial195.stdout: "2(4)", 2026-02-20T18:04:42.502 INFO:tasks.workunit.client.0.trial195.stdout: "3(0)", 2026-02-20T18:04:42.502 INFO:tasks.workunit.client.0.trial195.stdout: "4(2)" 2026-02-20T18:04:42.502 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:04:42.502 INFO:tasks.workunit.client.0.trial195.stdout: "info": { 2026-02-20T18:04:42.502 INFO:tasks.workunit.client.0.trial195.stdout: "pgid": "2.0s0", 2026-02-20T18:04:42.502 INFO:tasks.workunit.client.0.trial195.stdout: "shard": "0", 2026-02-20T18:04:42.502 INFO:tasks.workunit.client.0.trial195.stdout: "last_update": "48'101", 2026-02-20T18:04:42.502 INFO:tasks.workunit.client.0.trial195.stdout: "last_complete": "48'75", 2026-02-20T18:04:42.502 INFO:tasks.workunit.client.0.trial195.stdout: "log_tail": "0'0", 2026-02-20T18:04:42.502 INFO:tasks.workunit.client.0.trial195.stdout: "last_user_version": 101, 2026-02-20T18:04:42.502 INFO:tasks.workunit.client.0.trial195.stdout: "last_backfill": "MAX", 2026-02-20T18:04:42.502 INFO:tasks.workunit.client.0.trial195.stdout: "partial_writes_last_complete": [], 2026-02-20T18:04:42.503 INFO:tasks.workunit.client.0.trial195.stdout: "partial_writes_last_complete_epoch": "54", 2026-02-20T18:04:42.503 INFO:tasks.workunit.client.0.trial195.stdout: "purged_snaps": [], 2026-02-20T18:04:42.503 INFO:tasks.workunit.client.0.trial195.stdout: "history": { 2026-02-20T18:04:42.503 INFO:tasks.workunit.client.0.trial195.stdout: "epoch_created": 41, 2026-02-20T18:04:42.503 INFO:tasks.workunit.client.0.trial195.stdout: "epoch_pool_created": 41, 2026-02-20T18:04:42.503 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_started": 55, 2026-02-20T18:04:42.503 INFO:tasks.workunit.client.0.trial195.stdout: "last_interval_started": 54, 2026-02-20T18:04:42.503 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_clean": 43, 2026-02-20T18:04:42.503 INFO:tasks.workunit.client.0.trial195.stdout: "last_interval_clean": 41, 2026-02-20T18:04:42.503 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_split": 0, 2026-02-20T18:04:42.503 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_marked_full": 0, 2026-02-20T18:04:42.504 INFO:tasks.workunit.client.0.trial195.stdout: "same_up_since": 54, 2026-02-20T18:04:42.504 INFO:tasks.workunit.client.0.trial195.stdout: "same_interval_since": 54, 2026-02-20T18:04:42.504 INFO:tasks.workunit.client.0.trial195.stdout: "same_primary_since": 41, 2026-02-20T18:04:42.504 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub": "0'0", 2026-02-20T18:04:42.504 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub_stamp": "2026-02-20T18:04:14.921735+0000", 2026-02-20T18:04:42.504 INFO:tasks.workunit.client.0.trial195.stdout: "last_deep_scrub": "0'0", 2026-02-20T18:04:42.504 INFO:tasks.workunit.client.0.trial195.stdout: "last_deep_scrub_stamp": "2026-02-20T18:04:14.921735+0000", 2026-02-20T18:04:42.504 INFO:tasks.workunit.client.0.trial195.stdout: "last_clean_scrub_stamp": "2026-02-20T18:04:14.921735+0000", 2026-02-20T18:04:42.504 INFO:tasks.workunit.client.0.trial195.stdout: "prior_readable_until_ub": 0 2026-02-20T18:04:42.504 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:04:42.504 INFO:tasks.workunit.client.0.trial195.stdout: "stats": { 2026-02-20T18:04:42.504 INFO:tasks.workunit.client.0.trial195.stdout: "version": "48'101", 2026-02-20T18:04:42.505 INFO:tasks.workunit.client.0.trial195.stdout: "reported_seq": 365, 2026-02-20T18:04:42.505 INFO:tasks.workunit.client.0.trial195.stdout: "reported_epoch": 55, 2026-02-20T18:04:42.505 INFO:tasks.workunit.client.0.trial195.stdout: "state": "active+recovery_unfound+degraded", 2026-02-20T18:04:42.505 INFO:tasks.workunit.client.0.trial195.stdout: "last_fresh": "2026-02-20T18:04:26.952389+0000", 2026-02-20T18:04:42.505 INFO:tasks.workunit.client.0.trial195.stdout: "last_change": "2026-02-20T18:04:26.952389+0000", 2026-02-20T18:04:42.505 INFO:tasks.workunit.client.0.trial195.stdout: "last_active": "2026-02-20T18:04:26.952389+0000", 2026-02-20T18:04:42.505 INFO:tasks.workunit.client.0.trial195.stdout: "last_peered": "2026-02-20T18:04:26.952389+0000", 2026-02-20T18:04:42.505 INFO:tasks.workunit.client.0.trial195.stdout: "last_clean": "2026-02-20T18:04:21.320608+0000", 2026-02-20T18:04:42.505 INFO:tasks.workunit.client.0.trial195.stdout: "last_became_active": "2026-02-20T18:04:26.950942+0000", 2026-02-20T18:04:42.505 INFO:tasks.workunit.client.0.trial195.stdout: "last_became_peered": "2026-02-20T18:04:26.950942+0000", 2026-02-20T18:04:42.506 INFO:tasks.workunit.client.0.trial195.stdout: "last_unstale": "2026-02-20T18:04:26.952389+0000", 2026-02-20T18:04:42.506 INFO:tasks.workunit.client.0.trial195.stdout: "last_undegraded": "2026-02-20T18:04:26.949730+0000", 2026-02-20T18:04:42.506 INFO:tasks.workunit.client.0.trial195.stdout: "last_fullsized": "2026-02-20T18:04:26.952389+0000", 2026-02-20T18:04:42.506 INFO:tasks.workunit.client.0.trial195.stdout: "mapping_epoch": 54, 2026-02-20T18:04:42.506 INFO:tasks.workunit.client.0.trial195.stdout: "log_start": "0'0", 2026-02-20T18:04:42.506 INFO:tasks.workunit.client.0.trial195.stdout: "ondisk_log_start": "0'0", 2026-02-20T18:04:42.506 INFO:tasks.workunit.client.0.trial195.stdout: "created": 41, 2026-02-20T18:04:42.506 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_clean": 43, 2026-02-20T18:04:42.506 INFO:tasks.workunit.client.0.trial195.stdout: "parent": "0.0", 2026-02-20T18:04:42.506 INFO:tasks.workunit.client.0.trial195.stdout: "parent_split_bits": 0, 2026-02-20T18:04:42.506 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub": "0'0", 2026-02-20T18:04:42.506 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub_stamp": "2026-02-20T18:04:14.921735+0000", 2026-02-20T18:04:42.507 INFO:tasks.workunit.client.0.trial195.stdout: "last_deep_scrub": "0'0", 2026-02-20T18:04:42.507 INFO:tasks.workunit.client.0.trial195.stdout: "last_deep_scrub_stamp": "2026-02-20T18:04:14.921735+0000", 2026-02-20T18:04:42.507 INFO:tasks.workunit.client.0.trial195.stdout: "last_clean_scrub_stamp": "2026-02-20T18:04:14.921735+0000", 2026-02-20T18:04:42.507 INFO:tasks.workunit.client.0.trial195.stdout: "objects_scrubbed": 0, 2026-02-20T18:04:42.507 INFO:tasks.workunit.client.0.trial195.stdout: "log_size": 101, 2026-02-20T18:04:42.507 INFO:tasks.workunit.client.0.trial195.stdout: "log_dups_size": 0, 2026-02-20T18:04:42.507 INFO:tasks.workunit.client.0.trial195.stdout: "ondisk_log_size": 101, 2026-02-20T18:04:42.507 INFO:tasks.workunit.client.0.trial195.stdout: "stats_invalid": false, 2026-02-20T18:04:42.507 INFO:tasks.workunit.client.0.trial195.stdout: "dirty_stats_invalid": false, 2026-02-20T18:04:42.507 INFO:tasks.workunit.client.0.trial195.stdout: "omap_stats_invalid": false, 2026-02-20T18:04:42.507 INFO:tasks.workunit.client.0.trial195.stdout: "hitset_stats_invalid": false, 2026-02-20T18:04:42.507 INFO:tasks.workunit.client.0.trial195.stdout: "hitset_bytes_stats_invalid": false, 2026-02-20T18:04:42.508 INFO:tasks.workunit.client.0.trial195.stdout: "pin_stats_invalid": false, 2026-02-20T18:04:42.508 INFO:tasks.workunit.client.0.trial195.stdout: "manifest_stats_invalid": false, 2026-02-20T18:04:42.508 INFO:tasks.workunit.client.0.trial195.stdout: "snaptrimq_len": 0, 2026-02-20T18:04:42.508 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub_duration": 0, 2026-02-20T18:04:42.508 INFO:tasks.workunit.client.0.trial195.stdout: "scrub_schedule": "no scrub is scheduled", 2026-02-20T18:04:42.508 INFO:tasks.workunit.client.0.trial195.stdout: "scrub_duration": 0, 2026-02-20T18:04:42.508 INFO:tasks.workunit.client.0.trial195.stdout: "objects_trimmed": 0, 2026-02-20T18:04:42.508 INFO:tasks.workunit.client.0.trial195.stdout: "snaptrim_duration": 0, 2026-02-20T18:04:42.508 INFO:tasks.workunit.client.0.trial195.stdout: "stat_sum": { 2026-02-20T18:04:42.508 INFO:tasks.workunit.client.0.trial195.stdout: "num_bytes": 413696, 2026-02-20T18:04:42.508 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects": 101, 2026-02-20T18:04:42.508 INFO:tasks.workunit.client.0.trial195.stdout: "num_object_clones": 0, 2026-02-20T18:04:42.508 INFO:tasks.workunit.client.0.trial195.stdout: "num_object_copies": 505, 2026-02-20T18:04:42.509 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_missing_on_primary": 1, 2026-02-20T18:04:42.509 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_missing": 1, 2026-02-20T18:04:42.509 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_degraded": 3, 2026-02-20T18:04:42.509 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_misplaced": 0, 2026-02-20T18:04:42.509 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_unfound": 1, 2026-02-20T18:04:42.509 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_dirty": 101, 2026-02-20T18:04:42.509 INFO:tasks.workunit.client.0.trial195.stdout: "num_whiteouts": 0, 2026-02-20T18:04:42.509 INFO:tasks.workunit.client.0.trial195.stdout: "num_read": 0, 2026-02-20T18:04:42.509 INFO:tasks.workunit.client.0.trial195.stdout: "num_read_kb": 0, 2026-02-20T18:04:42.509 INFO:tasks.workunit.client.0.trial195.stdout: "num_write": 101, 2026-02-20T18:04:42.509 INFO:tasks.workunit.client.0.trial195.stdout: "num_write_kb": 404, 2026-02-20T18:04:42.509 INFO:tasks.workunit.client.0.trial195.stdout: "num_scrub_errors": 0, 2026-02-20T18:04:42.509 INFO:tasks.workunit.client.0.trial195.stdout: "num_shallow_scrub_errors": 0, 2026-02-20T18:04:42.510 INFO:tasks.workunit.client.0.trial195.stdout: "num_deep_scrub_errors": 0, 2026-02-20T18:04:42.510 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_recovered": 0, 2026-02-20T18:04:42.510 INFO:tasks.workunit.client.0.trial195.stdout: "num_bytes_recovered": 0, 2026-02-20T18:04:42.510 INFO:tasks.workunit.client.0.trial195.stdout: "num_keys_recovered": 0, 2026-02-20T18:04:42.510 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_omap": 0, 2026-02-20T18:04:42.510 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_hit_set_archive": 0, 2026-02-20T18:04:42.510 INFO:tasks.workunit.client.0.trial195.stdout: "num_bytes_hit_set_archive": 0, 2026-02-20T18:04:42.510 INFO:tasks.workunit.client.0.trial195.stdout: "num_flush": 0, 2026-02-20T18:04:42.510 INFO:tasks.workunit.client.0.trial195.stdout: "num_flush_kb": 0, 2026-02-20T18:04:42.510 INFO:tasks.workunit.client.0.trial195.stdout: "num_evict": 0, 2026-02-20T18:04:42.510 INFO:tasks.workunit.client.0.trial195.stdout: "num_evict_kb": 0, 2026-02-20T18:04:42.510 INFO:tasks.workunit.client.0.trial195.stdout: "num_promote": 0, 2026-02-20T18:04:42.511 INFO:tasks.workunit.client.0.trial195.stdout: "num_flush_mode_high": 0, 2026-02-20T18:04:42.511 INFO:tasks.workunit.client.0.trial195.stdout: "num_flush_mode_low": 0, 2026-02-20T18:04:42.511 INFO:tasks.workunit.client.0.trial195.stdout: "num_evict_mode_some": 0, 2026-02-20T18:04:42.511 INFO:tasks.workunit.client.0.trial195.stdout: "num_evict_mode_full": 0, 2026-02-20T18:04:42.511 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_pinned": 0, 2026-02-20T18:04:42.511 INFO:tasks.workunit.client.0.trial195.stdout: "num_legacy_snapsets": 0, 2026-02-20T18:04:42.511 INFO:tasks.workunit.client.0.trial195.stdout: "num_large_omap_objects": 0, 2026-02-20T18:04:42.511 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_manifest": 0, 2026-02-20T18:04:42.511 INFO:tasks.workunit.client.0.trial195.stdout: "num_omap_bytes": 0, 2026-02-20T18:04:42.511 INFO:tasks.workunit.client.0.trial195.stdout: "num_omap_keys": 0, 2026-02-20T18:04:42.511 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_repaired": 0 2026-02-20T18:04:42.511 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:04:42.511 INFO:tasks.workunit.client.0.trial195.stdout: "up": [ 2026-02-20T18:04:42.512 INFO:tasks.workunit.client.0.trial195.stdout: 3, 2026-02-20T18:04:42.512 INFO:tasks.workunit.client.0.trial195.stdout: 1, 2026-02-20T18:04:42.512 INFO:tasks.workunit.client.0.trial195.stdout: 4, 2026-02-20T18:04:42.512 INFO:tasks.workunit.client.0.trial195.stdout: 0, 2026-02-20T18:04:42.512 INFO:tasks.workunit.client.0.trial195.stdout: 2 2026-02-20T18:04:42.512 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:04:42.512 INFO:tasks.workunit.client.0.trial195.stdout: "acting": [ 2026-02-20T18:04:42.512 INFO:tasks.workunit.client.0.trial195.stdout: 3, 2026-02-20T18:04:42.512 INFO:tasks.workunit.client.0.trial195.stdout: 1, 2026-02-20T18:04:42.512 INFO:tasks.workunit.client.0.trial195.stdout: 4, 2026-02-20T18:04:42.512 INFO:tasks.workunit.client.0.trial195.stdout: 0, 2026-02-20T18:04:42.512 INFO:tasks.workunit.client.0.trial195.stdout: 2 2026-02-20T18:04:42.513 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:04:42.513 INFO:tasks.workunit.client.0.trial195.stdout: "avail_no_missing": [ 2026-02-20T18:04:42.513 INFO:tasks.workunit.client.0.trial195.stdout: "0(3)", 2026-02-20T18:04:42.513 INFO:tasks.workunit.client.0.trial195.stdout: "4(2)" 2026-02-20T18:04:42.513 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:04:42.513 INFO:tasks.workunit.client.0.trial195.stdout: "object_location_counts": [ 2026-02-20T18:04:42.513 INFO:tasks.workunit.client.0.trial195.stdout: { 2026-02-20T18:04:42.513 INFO:tasks.workunit.client.0.trial195.stdout: "shards": "0(3),1(1),2(4),3(0),4(2)", 2026-02-20T18:04:42.513 INFO:tasks.workunit.client.0.trial195.stdout: "objects": 100 2026-02-20T18:04:42.513 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:04:42.513 INFO:tasks.workunit.client.0.trial195.stdout: { 2026-02-20T18:04:42.513 INFO:tasks.workunit.client.0.trial195.stdout: "shards": "0(3),4(2)", 2026-02-20T18:04:42.513 INFO:tasks.workunit.client.0.trial195.stdout: "objects": 1 2026-02-20T18:04:42.514 INFO:tasks.workunit.client.0.trial195.stdout: } 2026-02-20T18:04:42.514 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:04:42.514 INFO:tasks.workunit.client.0.trial195.stdout: "blocked_by": [], 2026-02-20T18:04:42.514 INFO:tasks.workunit.client.0.trial195.stdout: "up_primary": 3, 2026-02-20T18:04:42.514 INFO:tasks.workunit.client.0.trial195.stdout: "acting_primary": 3, 2026-02-20T18:04:42.514 INFO:tasks.workunit.client.0.trial195.stdout: "purged_snaps": [] 2026-02-20T18:04:42.514 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:04:42.514 INFO:tasks.workunit.client.0.trial195.stdout: "empty": 0, 2026-02-20T18:04:42.514 INFO:tasks.workunit.client.0.trial195.stdout: "dne": 0, 2026-02-20T18:04:42.514 INFO:tasks.workunit.client.0.trial195.stdout: "incomplete": 0, 2026-02-20T18:04:42.514 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_started": 55, 2026-02-20T18:04:42.514 INFO:tasks.workunit.client.0.trial195.stdout: "hit_set_history": { 2026-02-20T18:04:42.514 INFO:tasks.workunit.client.0.trial195.stdout: "current_last_update": "0'0", 2026-02-20T18:04:42.515 INFO:tasks.workunit.client.0.trial195.stdout: "history": [] 2026-02-20T18:04:42.515 INFO:tasks.workunit.client.0.trial195.stdout: } 2026-02-20T18:04:42.515 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:04:42.515 INFO:tasks.workunit.client.0.trial195.stdout: "peer_info": [ 2026-02-20T18:04:42.515 INFO:tasks.workunit.client.0.trial195.stdout: { 2026-02-20T18:04:42.515 INFO:tasks.workunit.client.0.trial195.stdout: "peer": "0(3)", 2026-02-20T18:04:42.515 INFO:tasks.workunit.client.0.trial195.stdout: "pgid": "2.0s3", 2026-02-20T18:04:42.515 INFO:tasks.workunit.client.0.trial195.stdout: "shard": "3", 2026-02-20T18:04:42.515 INFO:tasks.workunit.client.0.trial195.stdout: "last_update": "48'101", 2026-02-20T18:04:42.515 INFO:tasks.workunit.client.0.trial195.stdout: "last_complete": "48'101", 2026-02-20T18:04:42.516 INFO:tasks.workunit.client.0.trial195.stdout: "log_tail": "0'0", 2026-02-20T18:04:42.516 INFO:tasks.workunit.client.0.trial195.stdout: "last_user_version": 101, 2026-02-20T18:04:42.516 INFO:tasks.workunit.client.0.trial195.stdout: "last_backfill": "MAX", 2026-02-20T18:04:42.516 INFO:tasks.workunit.client.0.trial195.stdout: "partial_writes_last_complete": [], 2026-02-20T18:04:42.516 INFO:tasks.workunit.client.0.trial195.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-20T18:04:42.516 INFO:tasks.workunit.client.0.trial195.stdout: "purged_snaps": [], 2026-02-20T18:04:42.516 INFO:tasks.workunit.client.0.trial195.stdout: "history": { 2026-02-20T18:04:42.516 INFO:tasks.workunit.client.0.trial195.stdout: "epoch_created": 41, 2026-02-20T18:04:42.516 INFO:tasks.workunit.client.0.trial195.stdout: "epoch_pool_created": 41, 2026-02-20T18:04:42.516 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_started": 55, 2026-02-20T18:04:42.516 INFO:tasks.workunit.client.0.trial195.stdout: "last_interval_started": 54, 2026-02-20T18:04:42.516 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_clean": 43, 2026-02-20T18:04:42.516 INFO:tasks.workunit.client.0.trial195.stdout: "last_interval_clean": 41, 2026-02-20T18:04:42.517 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_split": 0, 2026-02-20T18:04:42.517 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_marked_full": 0, 2026-02-20T18:04:42.517 INFO:tasks.workunit.client.0.trial195.stdout: "same_up_since": 54, 2026-02-20T18:04:42.517 INFO:tasks.workunit.client.0.trial195.stdout: "same_interval_since": 54, 2026-02-20T18:04:42.517 INFO:tasks.workunit.client.0.trial195.stdout: "same_primary_since": 41, 2026-02-20T18:04:42.517 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub": "0'0", 2026-02-20T18:04:42.517 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub_stamp": "2026-02-20T18:04:14.921735+0000", 2026-02-20T18:04:42.517 INFO:tasks.workunit.client.0.trial195.stdout: "last_deep_scrub": "0'0", 2026-02-20T18:04:42.517 INFO:tasks.workunit.client.0.trial195.stdout: "last_deep_scrub_stamp": "2026-02-20T18:04:14.921735+0000", 2026-02-20T18:04:42.517 INFO:tasks.workunit.client.0.trial195.stdout: "last_clean_scrub_stamp": "2026-02-20T18:04:14.921735+0000", 2026-02-20T18:04:42.517 INFO:tasks.workunit.client.0.trial195.stdout: "prior_readable_until_ub": 0 2026-02-20T18:04:42.517 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:04:42.517 INFO:tasks.workunit.client.0.trial195.stdout: "stats": { 2026-02-20T18:04:42.518 INFO:tasks.workunit.client.0.trial195.stdout: "version": "48'101", 2026-02-20T18:04:42.518 INFO:tasks.workunit.client.0.trial195.stdout: "reported_seq": 353, 2026-02-20T18:04:42.518 INFO:tasks.workunit.client.0.trial195.stdout: "reported_epoch": 54, 2026-02-20T18:04:42.518 INFO:tasks.workunit.client.0.trial195.stdout: "state": "active+undersized+degraded", 2026-02-20T18:04:42.518 INFO:tasks.workunit.client.0.trial195.stdout: "last_fresh": "2026-02-20T18:04:24.279491+0000", 2026-02-20T18:04:42.518 INFO:tasks.workunit.client.0.trial195.stdout: "last_change": "2026-02-20T18:04:21.673339+0000", 2026-02-20T18:04:42.518 INFO:tasks.workunit.client.0.trial195.stdout: "last_active": "2026-02-20T18:04:24.279491+0000", 2026-02-20T18:04:42.518 INFO:tasks.workunit.client.0.trial195.stdout: "last_peered": "2026-02-20T18:04:24.279491+0000", 2026-02-20T18:04:42.518 INFO:tasks.workunit.client.0.trial195.stdout: "last_clean": "2026-02-20T18:04:21.320608+0000", 2026-02-20T18:04:42.518 INFO:tasks.workunit.client.0.trial195.stdout: "last_became_active": "2026-02-20T18:04:21.673339+0000", 2026-02-20T18:04:42.518 INFO:tasks.workunit.client.0.trial195.stdout: "last_became_peered": "2026-02-20T18:04:21.673339+0000", 2026-02-20T18:04:42.518 INFO:tasks.workunit.client.0.trial195.stdout: "last_unstale": "2026-02-20T18:04:24.279491+0000", 2026-02-20T18:04:42.519 INFO:tasks.workunit.client.0.trial195.stdout: "last_undegraded": "2026-02-20T18:04:21.672159+0000", 2026-02-20T18:04:42.519 INFO:tasks.workunit.client.0.trial195.stdout: "last_fullsized": "2026-02-20T18:04:21.672055+0000", 2026-02-20T18:04:42.519 INFO:tasks.workunit.client.0.trial195.stdout: "mapping_epoch": 54, 2026-02-20T18:04:42.519 INFO:tasks.workunit.client.0.trial195.stdout: "log_start": "0'0", 2026-02-20T18:04:42.519 INFO:tasks.workunit.client.0.trial195.stdout: "ondisk_log_start": "0'0", 2026-02-20T18:04:42.519 INFO:tasks.workunit.client.0.trial195.stdout: "created": 41, 2026-02-20T18:04:42.519 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_clean": 43, 2026-02-20T18:04:42.519 INFO:tasks.workunit.client.0.trial195.stdout: "parent": "0.0", 2026-02-20T18:04:42.519 INFO:tasks.workunit.client.0.trial195.stdout: "parent_split_bits": 0, 2026-02-20T18:04:42.519 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub": "0'0", 2026-02-20T18:04:42.519 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub_stamp": "2026-02-20T18:04:14.921735+0000", 2026-02-20T18:04:42.519 INFO:tasks.workunit.client.0.trial195.stdout: "last_deep_scrub": "0'0", 2026-02-20T18:04:42.520 INFO:tasks.workunit.client.0.trial195.stdout: "last_deep_scrub_stamp": "2026-02-20T18:04:14.921735+0000", 2026-02-20T18:04:42.520 INFO:tasks.workunit.client.0.trial195.stdout: "last_clean_scrub_stamp": "2026-02-20T18:04:14.921735+0000", 2026-02-20T18:04:42.520 INFO:tasks.workunit.client.0.trial195.stdout: "objects_scrubbed": 0, 2026-02-20T18:04:42.520 INFO:tasks.workunit.client.0.trial195.stdout: "log_size": 101, 2026-02-20T18:04:42.520 INFO:tasks.workunit.client.0.trial195.stdout: "log_dups_size": 0, 2026-02-20T18:04:42.520 INFO:tasks.workunit.client.0.trial195.stdout: "ondisk_log_size": 101, 2026-02-20T18:04:42.520 INFO:tasks.workunit.client.0.trial195.stdout: "stats_invalid": false, 2026-02-20T18:04:42.520 INFO:tasks.workunit.client.0.trial195.stdout: "dirty_stats_invalid": false, 2026-02-20T18:04:42.520 INFO:tasks.workunit.client.0.trial195.stdout: "omap_stats_invalid": false, 2026-02-20T18:04:42.520 INFO:tasks.workunit.client.0.trial195.stdout: "hitset_stats_invalid": false, 2026-02-20T18:04:42.520 INFO:tasks.workunit.client.0.trial195.stdout: "hitset_bytes_stats_invalid": false, 2026-02-20T18:04:42.520 INFO:tasks.workunit.client.0.trial195.stdout: "pin_stats_invalid": false, 2026-02-20T18:04:42.521 INFO:tasks.workunit.client.0.trial195.stdout: "manifest_stats_invalid": false, 2026-02-20T18:04:42.521 INFO:tasks.workunit.client.0.trial195.stdout: "snaptrimq_len": 0, 2026-02-20T18:04:42.521 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub_duration": 0, 2026-02-20T18:04:42.521 INFO:tasks.workunit.client.0.trial195.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-21T20:50:09.597014+0000", 2026-02-20T18:04:42.521 INFO:tasks.workunit.client.0.trial195.stdout: "scrub_duration": 0, 2026-02-20T18:04:42.521 INFO:tasks.workunit.client.0.trial195.stdout: "objects_trimmed": 0, 2026-02-20T18:04:42.521 INFO:tasks.workunit.client.0.trial195.stdout: "snaptrim_duration": 0, 2026-02-20T18:04:42.521 INFO:tasks.workunit.client.0.trial195.stdout: "stat_sum": { 2026-02-20T18:04:42.521 INFO:tasks.workunit.client.0.trial195.stdout: "num_bytes": 413696, 2026-02-20T18:04:42.521 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects": 101, 2026-02-20T18:04:42.521 INFO:tasks.workunit.client.0.trial195.stdout: "num_object_clones": 0, 2026-02-20T18:04:42.521 INFO:tasks.workunit.client.0.trial195.stdout: "num_object_copies": 505, 2026-02-20T18:04:42.521 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_missing_on_primary": 0, 2026-02-20T18:04:42.522 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_missing": 0, 2026-02-20T18:04:42.522 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_degraded": 101, 2026-02-20T18:04:42.522 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_misplaced": 0, 2026-02-20T18:04:42.522 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_unfound": 0, 2026-02-20T18:04:42.522 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_dirty": 101, 2026-02-20T18:04:42.522 INFO:tasks.workunit.client.0.trial195.stdout: "num_whiteouts": 0, 2026-02-20T18:04:42.522 INFO:tasks.workunit.client.0.trial195.stdout: "num_read": 0, 2026-02-20T18:04:42.522 INFO:tasks.workunit.client.0.trial195.stdout: "num_read_kb": 0, 2026-02-20T18:04:42.522 INFO:tasks.workunit.client.0.trial195.stdout: "num_write": 101, 2026-02-20T18:04:42.522 INFO:tasks.workunit.client.0.trial195.stdout: "num_write_kb": 404, 2026-02-20T18:04:42.522 INFO:tasks.workunit.client.0.trial195.stdout: "num_scrub_errors": 0, 2026-02-20T18:04:42.522 INFO:tasks.workunit.client.0.trial195.stdout: "num_shallow_scrub_errors": 0, 2026-02-20T18:04:42.522 INFO:tasks.workunit.client.0.trial195.stdout: "num_deep_scrub_errors": 0, 2026-02-20T18:04:42.523 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_recovered": 0, 2026-02-20T18:04:42.523 INFO:tasks.workunit.client.0.trial195.stdout: "num_bytes_recovered": 0, 2026-02-20T18:04:42.523 INFO:tasks.workunit.client.0.trial195.stdout: "num_keys_recovered": 0, 2026-02-20T18:04:42.523 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_omap": 0, 2026-02-20T18:04:42.523 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_hit_set_archive": 0, 2026-02-20T18:04:42.523 INFO:tasks.workunit.client.0.trial195.stdout: "num_bytes_hit_set_archive": 0, 2026-02-20T18:04:42.523 INFO:tasks.workunit.client.0.trial195.stdout: "num_flush": 0, 2026-02-20T18:04:42.523 INFO:tasks.workunit.client.0.trial195.stdout: "num_flush_kb": 0, 2026-02-20T18:04:42.523 INFO:tasks.workunit.client.0.trial195.stdout: "num_evict": 0, 2026-02-20T18:04:42.523 INFO:tasks.workunit.client.0.trial195.stdout: "num_evict_kb": 0, 2026-02-20T18:04:42.523 INFO:tasks.workunit.client.0.trial195.stdout: "num_promote": 0, 2026-02-20T18:04:42.523 INFO:tasks.workunit.client.0.trial195.stdout: "num_flush_mode_high": 0, 2026-02-20T18:04:42.523 INFO:tasks.workunit.client.0.trial195.stdout: "num_flush_mode_low": 0, 2026-02-20T18:04:42.524 INFO:tasks.workunit.client.0.trial195.stdout: "num_evict_mode_some": 0, 2026-02-20T18:04:42.524 INFO:tasks.workunit.client.0.trial195.stdout: "num_evict_mode_full": 0, 2026-02-20T18:04:42.524 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_pinned": 0, 2026-02-20T18:04:42.524 INFO:tasks.workunit.client.0.trial195.stdout: "num_legacy_snapsets": 0, 2026-02-20T18:04:42.524 INFO:tasks.workunit.client.0.trial195.stdout: "num_large_omap_objects": 0, 2026-02-20T18:04:42.524 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_manifest": 0, 2026-02-20T18:04:42.524 INFO:tasks.workunit.client.0.trial195.stdout: "num_omap_bytes": 0, 2026-02-20T18:04:42.524 INFO:tasks.workunit.client.0.trial195.stdout: "num_omap_keys": 0, 2026-02-20T18:04:42.524 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_repaired": 0 2026-02-20T18:04:42.524 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:04:42.524 INFO:tasks.workunit.client.0.trial195.stdout: "up": [ 2026-02-20T18:04:42.524 INFO:tasks.workunit.client.0.trial195.stdout: 3, 2026-02-20T18:04:42.524 INFO:tasks.workunit.client.0.trial195.stdout: 1, 2026-02-20T18:04:42.525 INFO:tasks.workunit.client.0.trial195.stdout: 4, 2026-02-20T18:04:42.525 INFO:tasks.workunit.client.0.trial195.stdout: 0, 2026-02-20T18:04:42.525 INFO:tasks.workunit.client.0.trial195.stdout: 2 2026-02-20T18:04:42.525 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:04:42.525 INFO:tasks.workunit.client.0.trial195.stdout: "acting": [ 2026-02-20T18:04:42.525 INFO:tasks.workunit.client.0.trial195.stdout: 3, 2026-02-20T18:04:42.525 INFO:tasks.workunit.client.0.trial195.stdout: 1, 2026-02-20T18:04:42.525 INFO:tasks.workunit.client.0.trial195.stdout: 4, 2026-02-20T18:04:42.525 INFO:tasks.workunit.client.0.trial195.stdout: 0, 2026-02-20T18:04:42.525 INFO:tasks.workunit.client.0.trial195.stdout: 2 2026-02-20T18:04:42.525 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:04:42.526 INFO:tasks.workunit.client.0.trial195.stdout: "avail_no_missing": [ 2026-02-20T18:04:42.526 INFO:tasks.workunit.client.0.trial195.stdout: "3(0)", 2026-02-20T18:04:42.526 INFO:tasks.workunit.client.0.trial195.stdout: "0(3)", 2026-02-20T18:04:42.526 INFO:tasks.workunit.client.0.trial195.stdout: "1(1)", 2026-02-20T18:04:42.526 INFO:tasks.workunit.client.0.trial195.stdout: "4(2)" 2026-02-20T18:04:42.526 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:04:42.526 INFO:tasks.workunit.client.0.trial195.stdout: "object_location_counts": [ 2026-02-20T18:04:42.526 INFO:tasks.workunit.client.0.trial195.stdout: { 2026-02-20T18:04:42.526 INFO:tasks.workunit.client.0.trial195.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-20T18:04:42.526 INFO:tasks.workunit.client.0.trial195.stdout: "objects": 101 2026-02-20T18:04:42.526 INFO:tasks.workunit.client.0.trial195.stdout: } 2026-02-20T18:04:42.526 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:04:42.526 INFO:tasks.workunit.client.0.trial195.stdout: "blocked_by": [], 2026-02-20T18:04:42.527 INFO:tasks.workunit.client.0.trial195.stdout: "up_primary": 3, 2026-02-20T18:04:42.527 INFO:tasks.workunit.client.0.trial195.stdout: "acting_primary": 3, 2026-02-20T18:04:42.527 INFO:tasks.workunit.client.0.trial195.stdout: "purged_snaps": [] 2026-02-20T18:04:42.527 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:04:42.527 INFO:tasks.workunit.client.0.trial195.stdout: "empty": 0, 2026-02-20T18:04:42.527 INFO:tasks.workunit.client.0.trial195.stdout: "dne": 0, 2026-02-20T18:04:42.527 INFO:tasks.workunit.client.0.trial195.stdout: "incomplete": 0, 2026-02-20T18:04:42.527 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_started": 55, 2026-02-20T18:04:42.527 INFO:tasks.workunit.client.0.trial195.stdout: "hit_set_history": { 2026-02-20T18:04:42.527 INFO:tasks.workunit.client.0.trial195.stdout: "current_last_update": "0'0", 2026-02-20T18:04:42.527 INFO:tasks.workunit.client.0.trial195.stdout: "history": [] 2026-02-20T18:04:42.527 INFO:tasks.workunit.client.0.trial195.stdout: } 2026-02-20T18:04:42.527 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:04:42.528 INFO:tasks.workunit.client.0.trial195.stdout: { 2026-02-20T18:04:42.528 INFO:tasks.workunit.client.0.trial195.stdout: "peer": "1(1)", 2026-02-20T18:04:42.528 INFO:tasks.workunit.client.0.trial195.stdout: "pgid": "2.0s1", 2026-02-20T18:04:42.528 INFO:tasks.workunit.client.0.trial195.stdout: "shard": "1", 2026-02-20T18:04:42.528 INFO:tasks.workunit.client.0.trial195.stdout: "last_update": "48'101", 2026-02-20T18:04:42.528 INFO:tasks.workunit.client.0.trial195.stdout: "last_complete": "48'101", 2026-02-20T18:04:42.528 INFO:tasks.workunit.client.0.trial195.stdout: "log_tail": "0'0", 2026-02-20T18:04:42.528 INFO:tasks.workunit.client.0.trial195.stdout: "last_user_version": 101, 2026-02-20T18:04:42.528 INFO:tasks.workunit.client.0.trial195.stdout: "last_backfill": "MAX", 2026-02-20T18:04:42.528 INFO:tasks.workunit.client.0.trial195.stdout: "partial_writes_last_complete": [], 2026-02-20T18:04:42.528 INFO:tasks.workunit.client.0.trial195.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-20T18:04:42.528 INFO:tasks.workunit.client.0.trial195.stdout: "purged_snaps": [], 2026-02-20T18:04:42.528 INFO:tasks.workunit.client.0.trial195.stdout: "history": { 2026-02-20T18:04:42.529 INFO:tasks.workunit.client.0.trial195.stdout: "epoch_created": 41, 2026-02-20T18:04:42.529 INFO:tasks.workunit.client.0.trial195.stdout: "epoch_pool_created": 41, 2026-02-20T18:04:42.529 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_started": 55, 2026-02-20T18:04:42.529 INFO:tasks.workunit.client.0.trial195.stdout: "last_interval_started": 54, 2026-02-20T18:04:42.529 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_clean": 43, 2026-02-20T18:04:42.529 INFO:tasks.workunit.client.0.trial195.stdout: "last_interval_clean": 41, 2026-02-20T18:04:42.529 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_split": 0, 2026-02-20T18:04:42.529 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_marked_full": 0, 2026-02-20T18:04:42.529 INFO:tasks.workunit.client.0.trial195.stdout: "same_up_since": 54, 2026-02-20T18:04:42.529 INFO:tasks.workunit.client.0.trial195.stdout: "same_interval_since": 54, 2026-02-20T18:04:42.529 INFO:tasks.workunit.client.0.trial195.stdout: "same_primary_since": 41, 2026-02-20T18:04:42.529 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub": "0'0", 2026-02-20T18:04:42.529 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub_stamp": "2026-02-20T18:04:14.921735+0000", 2026-02-20T18:04:42.530 INFO:tasks.workunit.client.0.trial195.stdout: "last_deep_scrub": "0'0", 2026-02-20T18:04:42.530 INFO:tasks.workunit.client.0.trial195.stdout: "last_deep_scrub_stamp": "2026-02-20T18:04:14.921735+0000", 2026-02-20T18:04:42.530 INFO:tasks.workunit.client.0.trial195.stdout: "last_clean_scrub_stamp": "2026-02-20T18:04:14.921735+0000", 2026-02-20T18:04:42.530 INFO:tasks.workunit.client.0.trial195.stdout: "prior_readable_until_ub": 0 2026-02-20T18:04:42.530 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:04:42.530 INFO:tasks.workunit.client.0.trial195.stdout: "stats": { 2026-02-20T18:04:42.530 INFO:tasks.workunit.client.0.trial195.stdout: "version": "48'101", 2026-02-20T18:04:42.530 INFO:tasks.workunit.client.0.trial195.stdout: "reported_seq": 134, 2026-02-20T18:04:42.530 INFO:tasks.workunit.client.0.trial195.stdout: "reported_epoch": 48, 2026-02-20T18:04:42.530 INFO:tasks.workunit.client.0.trial195.stdout: "state": "active+undersized+degraded", 2026-02-20T18:04:42.530 INFO:tasks.workunit.client.0.trial195.stdout: "last_fresh": "2026-02-20T18:04:24.279491+0000", 2026-02-20T18:04:42.530 INFO:tasks.workunit.client.0.trial195.stdout: "last_change": "2026-02-20T18:04:21.673339+0000", 2026-02-20T18:04:42.530 INFO:tasks.workunit.client.0.trial195.stdout: "last_active": "2026-02-20T18:04:24.279491+0000", 2026-02-20T18:04:42.531 INFO:tasks.workunit.client.0.trial195.stdout: "last_peered": "2026-02-20T18:04:24.279491+0000", 2026-02-20T18:04:42.531 INFO:tasks.workunit.client.0.trial195.stdout: "last_clean": "2026-02-20T18:04:21.320608+0000", 2026-02-20T18:04:42.531 INFO:tasks.workunit.client.0.trial195.stdout: "last_became_active": "2026-02-20T18:04:21.673339+0000", 2026-02-20T18:04:42.531 INFO:tasks.workunit.client.0.trial195.stdout: "last_became_peered": "2026-02-20T18:04:21.673339+0000", 2026-02-20T18:04:42.531 INFO:tasks.workunit.client.0.trial195.stdout: "last_unstale": "2026-02-20T18:04:24.279491+0000", 2026-02-20T18:04:42.531 INFO:tasks.workunit.client.0.trial195.stdout: "last_undegraded": "2026-02-20T18:04:21.672159+0000", 2026-02-20T18:04:42.531 INFO:tasks.workunit.client.0.trial195.stdout: "last_fullsized": "2026-02-20T18:04:21.672055+0000", 2026-02-20T18:04:42.531 INFO:tasks.workunit.client.0.trial195.stdout: "mapping_epoch": 54, 2026-02-20T18:04:42.531 INFO:tasks.workunit.client.0.trial195.stdout: "log_start": "0'0", 2026-02-20T18:04:42.531 INFO:tasks.workunit.client.0.trial195.stdout: "ondisk_log_start": "0'0", 2026-02-20T18:04:42.531 INFO:tasks.workunit.client.0.trial195.stdout: "created": 41, 2026-02-20T18:04:42.531 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_clean": 43, 2026-02-20T18:04:42.531 INFO:tasks.workunit.client.0.trial195.stdout: "parent": "0.0", 2026-02-20T18:04:42.532 INFO:tasks.workunit.client.0.trial195.stdout: "parent_split_bits": 0, 2026-02-20T18:04:42.532 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub": "0'0", 2026-02-20T18:04:42.532 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub_stamp": "2026-02-20T18:04:14.921735+0000", 2026-02-20T18:04:42.532 INFO:tasks.workunit.client.0.trial195.stdout: "last_deep_scrub": "0'0", 2026-02-20T18:04:42.532 INFO:tasks.workunit.client.0.trial195.stdout: "last_deep_scrub_stamp": "2026-02-20T18:04:14.921735+0000", 2026-02-20T18:04:42.532 INFO:tasks.workunit.client.0.trial195.stdout: "last_clean_scrub_stamp": "2026-02-20T18:04:14.921735+0000", 2026-02-20T18:04:42.532 INFO:tasks.workunit.client.0.trial195.stdout: "objects_scrubbed": 0, 2026-02-20T18:04:42.532 INFO:tasks.workunit.client.0.trial195.stdout: "log_size": 101, 2026-02-20T18:04:42.532 INFO:tasks.workunit.client.0.trial195.stdout: "log_dups_size": 0, 2026-02-20T18:04:42.532 INFO:tasks.workunit.client.0.trial195.stdout: "ondisk_log_size": 101, 2026-02-20T18:04:42.532 INFO:tasks.workunit.client.0.trial195.stdout: "stats_invalid": false, 2026-02-20T18:04:42.532 INFO:tasks.workunit.client.0.trial195.stdout: "dirty_stats_invalid": false, 2026-02-20T18:04:42.532 INFO:tasks.workunit.client.0.trial195.stdout: "omap_stats_invalid": false, 2026-02-20T18:04:42.533 INFO:tasks.workunit.client.0.trial195.stdout: "hitset_stats_invalid": false, 2026-02-20T18:04:42.533 INFO:tasks.workunit.client.0.trial195.stdout: "hitset_bytes_stats_invalid": false, 2026-02-20T18:04:42.533 INFO:tasks.workunit.client.0.trial195.stdout: "pin_stats_invalid": false, 2026-02-20T18:04:42.533 INFO:tasks.workunit.client.0.trial195.stdout: "manifest_stats_invalid": false, 2026-02-20T18:04:42.533 INFO:tasks.workunit.client.0.trial195.stdout: "snaptrimq_len": 0, 2026-02-20T18:04:42.533 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub_duration": 0, 2026-02-20T18:04:42.533 INFO:tasks.workunit.client.0.trial195.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-21T20:50:09.597014+0000", 2026-02-20T18:04:42.533 INFO:tasks.workunit.client.0.trial195.stdout: "scrub_duration": 0, 2026-02-20T18:04:42.533 INFO:tasks.workunit.client.0.trial195.stdout: "objects_trimmed": 0, 2026-02-20T18:04:42.533 INFO:tasks.workunit.client.0.trial195.stdout: "snaptrim_duration": 0, 2026-02-20T18:04:42.533 INFO:tasks.workunit.client.0.trial195.stdout: "stat_sum": { 2026-02-20T18:04:42.533 INFO:tasks.workunit.client.0.trial195.stdout: "num_bytes": 413696, 2026-02-20T18:04:42.533 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects": 101, 2026-02-20T18:04:42.533 INFO:tasks.workunit.client.0.trial195.stdout: "num_object_clones": 0, 2026-02-20T18:04:42.533 INFO:tasks.workunit.client.0.trial195.stdout: "num_object_copies": 505, 2026-02-20T18:04:42.534 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_missing_on_primary": 0, 2026-02-20T18:04:42.534 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_missing": 1, 2026-02-20T18:04:42.534 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_degraded": 101, 2026-02-20T18:04:42.534 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_misplaced": 0, 2026-02-20T18:04:42.534 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_unfound": 0, 2026-02-20T18:04:42.534 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_dirty": 101, 2026-02-20T18:04:42.534 INFO:tasks.workunit.client.0.trial195.stdout: "num_whiteouts": 0, 2026-02-20T18:04:42.534 INFO:tasks.workunit.client.0.trial195.stdout: "num_read": 0, 2026-02-20T18:04:42.534 INFO:tasks.workunit.client.0.trial195.stdout: "num_read_kb": 0, 2026-02-20T18:04:42.534 INFO:tasks.workunit.client.0.trial195.stdout: "num_write": 101, 2026-02-20T18:04:42.534 INFO:tasks.workunit.client.0.trial195.stdout: "num_write_kb": 404, 2026-02-20T18:04:42.534 INFO:tasks.workunit.client.0.trial195.stdout: "num_scrub_errors": 0, 2026-02-20T18:04:42.534 INFO:tasks.workunit.client.0.trial195.stdout: "num_shallow_scrub_errors": 0, 2026-02-20T18:04:42.534 INFO:tasks.workunit.client.0.trial195.stdout: "num_deep_scrub_errors": 0, 2026-02-20T18:04:42.535 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_recovered": 0, 2026-02-20T18:04:42.535 INFO:tasks.workunit.client.0.trial195.stdout: "num_bytes_recovered": 0, 2026-02-20T18:04:42.535 INFO:tasks.workunit.client.0.trial195.stdout: "num_keys_recovered": 0, 2026-02-20T18:04:42.535 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_omap": 0, 2026-02-20T18:04:42.535 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_hit_set_archive": 0, 2026-02-20T18:04:42.535 INFO:tasks.workunit.client.0.trial195.stdout: "num_bytes_hit_set_archive": 0, 2026-02-20T18:04:42.535 INFO:tasks.workunit.client.0.trial195.stdout: "num_flush": 0, 2026-02-20T18:04:42.535 INFO:tasks.workunit.client.0.trial195.stdout: "num_flush_kb": 0, 2026-02-20T18:04:42.535 INFO:tasks.workunit.client.0.trial195.stdout: "num_evict": 0, 2026-02-20T18:04:42.535 INFO:tasks.workunit.client.0.trial195.stdout: "num_evict_kb": 0, 2026-02-20T18:04:42.535 INFO:tasks.workunit.client.0.trial195.stdout: "num_promote": 0, 2026-02-20T18:04:42.535 INFO:tasks.workunit.client.0.trial195.stdout: "num_flush_mode_high": 0, 2026-02-20T18:04:42.535 INFO:tasks.workunit.client.0.trial195.stdout: "num_flush_mode_low": 0, 2026-02-20T18:04:42.536 INFO:tasks.workunit.client.0.trial195.stdout: "num_evict_mode_some": 0, 2026-02-20T18:04:42.536 INFO:tasks.workunit.client.0.trial195.stdout: "num_evict_mode_full": 0, 2026-02-20T18:04:42.536 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_pinned": 0, 2026-02-20T18:04:42.536 INFO:tasks.workunit.client.0.trial195.stdout: "num_legacy_snapsets": 0, 2026-02-20T18:04:42.536 INFO:tasks.workunit.client.0.trial195.stdout: "num_large_omap_objects": 0, 2026-02-20T18:04:42.536 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_manifest": 0, 2026-02-20T18:04:42.536 INFO:tasks.workunit.client.0.trial195.stdout: "num_omap_bytes": 0, 2026-02-20T18:04:42.536 INFO:tasks.workunit.client.0.trial195.stdout: "num_omap_keys": 0, 2026-02-20T18:04:42.536 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_repaired": 0 2026-02-20T18:04:42.536 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:04:42.536 INFO:tasks.workunit.client.0.trial195.stdout: "up": [ 2026-02-20T18:04:42.536 INFO:tasks.workunit.client.0.trial195.stdout: 3, 2026-02-20T18:04:42.536 INFO:tasks.workunit.client.0.trial195.stdout: 1, 2026-02-20T18:04:42.536 INFO:tasks.workunit.client.0.trial195.stdout: 4, 2026-02-20T18:04:42.537 INFO:tasks.workunit.client.0.trial195.stdout: 0, 2026-02-20T18:04:42.537 INFO:tasks.workunit.client.0.trial195.stdout: 2 2026-02-20T18:04:42.537 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:04:42.537 INFO:tasks.workunit.client.0.trial195.stdout: "acting": [ 2026-02-20T18:04:42.537 INFO:tasks.workunit.client.0.trial195.stdout: 3, 2026-02-20T18:04:42.537 INFO:tasks.workunit.client.0.trial195.stdout: 1, 2026-02-20T18:04:42.537 INFO:tasks.workunit.client.0.trial195.stdout: 4, 2026-02-20T18:04:42.537 INFO:tasks.workunit.client.0.trial195.stdout: 0, 2026-02-20T18:04:42.537 INFO:tasks.workunit.client.0.trial195.stdout: 2 2026-02-20T18:04:42.537 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:04:42.537 INFO:tasks.workunit.client.0.trial195.stdout: "avail_no_missing": [ 2026-02-20T18:04:42.537 INFO:tasks.workunit.client.0.trial195.stdout: "3(0)", 2026-02-20T18:04:42.537 INFO:tasks.workunit.client.0.trial195.stdout: "0(3)", 2026-02-20T18:04:42.537 INFO:tasks.workunit.client.0.trial195.stdout: "1(1)", 2026-02-20T18:04:42.537 INFO:tasks.workunit.client.0.trial195.stdout: "4(2)" 2026-02-20T18:04:42.537 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:04:42.538 INFO:tasks.workunit.client.0.trial195.stdout: "object_location_counts": [ 2026-02-20T18:04:42.538 INFO:tasks.workunit.client.0.trial195.stdout: { 2026-02-20T18:04:42.538 INFO:tasks.workunit.client.0.trial195.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-20T18:04:42.538 INFO:tasks.workunit.client.0.trial195.stdout: "objects": 101 2026-02-20T18:04:42.538 INFO:tasks.workunit.client.0.trial195.stdout: } 2026-02-20T18:04:42.538 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:04:42.538 INFO:tasks.workunit.client.0.trial195.stdout: "blocked_by": [], 2026-02-20T18:04:42.538 INFO:tasks.workunit.client.0.trial195.stdout: "up_primary": 3, 2026-02-20T18:04:42.538 INFO:tasks.workunit.client.0.trial195.stdout: "acting_primary": 3, 2026-02-20T18:04:42.538 INFO:tasks.workunit.client.0.trial195.stdout: "purged_snaps": [] 2026-02-20T18:04:42.538 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:04:42.538 INFO:tasks.workunit.client.0.trial195.stdout: "empty": 0, 2026-02-20T18:04:42.538 INFO:tasks.workunit.client.0.trial195.stdout: "dne": 0, 2026-02-20T18:04:42.538 INFO:tasks.workunit.client.0.trial195.stdout: "incomplete": 0, 2026-02-20T18:04:42.538 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_started": 55, 2026-02-20T18:04:42.538 INFO:tasks.workunit.client.0.trial195.stdout: "hit_set_history": { 2026-02-20T18:04:42.538 INFO:tasks.workunit.client.0.trial195.stdout: "current_last_update": "0'0", 2026-02-20T18:04:42.539 INFO:tasks.workunit.client.0.trial195.stdout: "history": [] 2026-02-20T18:04:42.539 INFO:tasks.workunit.client.0.trial195.stdout: } 2026-02-20T18:04:42.539 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:04:42.539 INFO:tasks.workunit.client.0.trial195.stdout: { 2026-02-20T18:04:42.539 INFO:tasks.workunit.client.0.trial195.stdout: "peer": "2(4)", 2026-02-20T18:04:42.539 INFO:tasks.workunit.client.0.trial195.stdout: "pgid": "2.0s4", 2026-02-20T18:04:42.539 INFO:tasks.workunit.client.0.trial195.stdout: "shard": "4", 2026-02-20T18:04:42.539 INFO:tasks.workunit.client.0.trial195.stdout: "last_update": "48'101", 2026-02-20T18:04:42.539 INFO:tasks.workunit.client.0.trial195.stdout: "last_complete": "48'75", 2026-02-20T18:04:42.539 INFO:tasks.workunit.client.0.trial195.stdout: "log_tail": "0'0", 2026-02-20T18:04:42.539 INFO:tasks.workunit.client.0.trial195.stdout: "last_user_version": 101, 2026-02-20T18:04:42.539 INFO:tasks.workunit.client.0.trial195.stdout: "last_backfill": "MAX", 2026-02-20T18:04:42.539 INFO:tasks.workunit.client.0.trial195.stdout: "partial_writes_last_complete": [], 2026-02-20T18:04:42.539 INFO:tasks.workunit.client.0.trial195.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-20T18:04:42.539 INFO:tasks.workunit.client.0.trial195.stdout: "purged_snaps": [], 2026-02-20T18:04:42.540 INFO:tasks.workunit.client.0.trial195.stdout: "history": { 2026-02-20T18:04:42.540 INFO:tasks.workunit.client.0.trial195.stdout: "epoch_created": 41, 2026-02-20T18:04:42.540 INFO:tasks.workunit.client.0.trial195.stdout: "epoch_pool_created": 41, 2026-02-20T18:04:42.540 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_started": 55, 2026-02-20T18:04:42.540 INFO:tasks.workunit.client.0.trial195.stdout: "last_interval_started": 54, 2026-02-20T18:04:42.540 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_clean": 43, 2026-02-20T18:04:42.540 INFO:tasks.workunit.client.0.trial195.stdout: "last_interval_clean": 41, 2026-02-20T18:04:42.540 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_split": 0, 2026-02-20T18:04:42.540 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_marked_full": 0, 2026-02-20T18:04:42.540 INFO:tasks.workunit.client.0.trial195.stdout: "same_up_since": 54, 2026-02-20T18:04:42.540 INFO:tasks.workunit.client.0.trial195.stdout: "same_interval_since": 54, 2026-02-20T18:04:42.540 INFO:tasks.workunit.client.0.trial195.stdout: "same_primary_since": 41, 2026-02-20T18:04:42.540 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub": "0'0", 2026-02-20T18:04:42.540 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub_stamp": "2026-02-20T18:04:14.921735+0000", 2026-02-20T18:04:42.540 INFO:tasks.workunit.client.0.trial195.stdout: "last_deep_scrub": "0'0", 2026-02-20T18:04:42.540 INFO:tasks.workunit.client.0.trial195.stdout: "last_deep_scrub_stamp": "2026-02-20T18:04:14.921735+0000", 2026-02-20T18:04:42.541 INFO:tasks.workunit.client.0.trial195.stdout: "last_clean_scrub_stamp": "2026-02-20T18:04:14.921735+0000", 2026-02-20T18:04:42.541 INFO:tasks.workunit.client.0.trial195.stdout: "prior_readable_until_ub": 0 2026-02-20T18:04:42.541 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:04:42.541 INFO:tasks.workunit.client.0.trial195.stdout: "stats": { 2026-02-20T18:04:42.541 INFO:tasks.workunit.client.0.trial195.stdout: "version": "48'101", 2026-02-20T18:04:42.541 INFO:tasks.workunit.client.0.trial195.stdout: "reported_seq": 141, 2026-02-20T18:04:42.541 INFO:tasks.workunit.client.0.trial195.stdout: "reported_epoch": 52, 2026-02-20T18:04:42.541 INFO:tasks.workunit.client.0.trial195.stdout: "state": "remapped+peering", 2026-02-20T18:04:42.541 INFO:tasks.workunit.client.0.trial195.stdout: "last_fresh": "2026-02-20T18:04:25.763214+0000", 2026-02-20T18:04:42.541 INFO:tasks.workunit.client.0.trial195.stdout: "last_change": "2026-02-20T18:04:25.662374+0000", 2026-02-20T18:04:42.541 INFO:tasks.workunit.client.0.trial195.stdout: "last_active": "2026-02-20T18:04:25.662374+0000", 2026-02-20T18:04:42.541 INFO:tasks.workunit.client.0.trial195.stdout: "last_peered": "2026-02-20T18:04:25.558363+0000", 2026-02-20T18:04:42.541 INFO:tasks.workunit.client.0.trial195.stdout: "last_clean": "2026-02-20T18:04:21.320608+0000", 2026-02-20T18:04:42.541 INFO:tasks.workunit.client.0.trial195.stdout: "last_became_active": "2026-02-20T18:04:21.673339+0000", 2026-02-20T18:04:42.541 INFO:tasks.workunit.client.0.trial195.stdout: "last_became_peered": "2026-02-20T18:04:21.673339+0000", 2026-02-20T18:04:42.541 INFO:tasks.workunit.client.0.trial195.stdout: "last_unstale": "2026-02-20T18:04:25.763214+0000", 2026-02-20T18:04:42.541 INFO:tasks.workunit.client.0.trial195.stdout: "last_undegraded": "2026-02-20T18:04:25.763214+0000", 2026-02-20T18:04:42.542 INFO:tasks.workunit.client.0.trial195.stdout: "last_fullsized": "2026-02-20T18:04:25.763214+0000", 2026-02-20T18:04:42.542 INFO:tasks.workunit.client.0.trial195.stdout: "mapping_epoch": 54, 2026-02-20T18:04:42.542 INFO:tasks.workunit.client.0.trial195.stdout: "log_start": "0'0", 2026-02-20T18:04:42.542 INFO:tasks.workunit.client.0.trial195.stdout: "ondisk_log_start": "0'0", 2026-02-20T18:04:42.542 INFO:tasks.workunit.client.0.trial195.stdout: "created": 41, 2026-02-20T18:04:42.542 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_clean": 43, 2026-02-20T18:04:42.542 INFO:tasks.workunit.client.0.trial195.stdout: "parent": "0.0", 2026-02-20T18:04:42.542 INFO:tasks.workunit.client.0.trial195.stdout: "parent_split_bits": 0, 2026-02-20T18:04:42.542 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub": "0'0", 2026-02-20T18:04:42.542 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub_stamp": "2026-02-20T18:04:14.921735+0000", 2026-02-20T18:04:42.542 INFO:tasks.workunit.client.0.trial195.stdout: "last_deep_scrub": "0'0", 2026-02-20T18:04:42.542 INFO:tasks.workunit.client.0.trial195.stdout: "last_deep_scrub_stamp": "2026-02-20T18:04:14.921735+0000", 2026-02-20T18:04:42.542 INFO:tasks.workunit.client.0.trial195.stdout: "last_clean_scrub_stamp": "2026-02-20T18:04:14.921735+0000", 2026-02-20T18:04:42.542 INFO:tasks.workunit.client.0.trial195.stdout: "objects_scrubbed": 0, 2026-02-20T18:04:42.542 INFO:tasks.workunit.client.0.trial195.stdout: "log_size": 101, 2026-02-20T18:04:42.542 INFO:tasks.workunit.client.0.trial195.stdout: "log_dups_size": 0, 2026-02-20T18:04:42.543 INFO:tasks.workunit.client.0.trial195.stdout: "ondisk_log_size": 101, 2026-02-20T18:04:42.543 INFO:tasks.workunit.client.0.trial195.stdout: "stats_invalid": false, 2026-02-20T18:04:42.543 INFO:tasks.workunit.client.0.trial195.stdout: "dirty_stats_invalid": false, 2026-02-20T18:04:42.543 INFO:tasks.workunit.client.0.trial195.stdout: "omap_stats_invalid": false, 2026-02-20T18:04:42.543 INFO:tasks.workunit.client.0.trial195.stdout: "hitset_stats_invalid": false, 2026-02-20T18:04:42.543 INFO:tasks.workunit.client.0.trial195.stdout: "hitset_bytes_stats_invalid": false, 2026-02-20T18:04:42.543 INFO:tasks.workunit.client.0.trial195.stdout: "pin_stats_invalid": false, 2026-02-20T18:04:42.543 INFO:tasks.workunit.client.0.trial195.stdout: "manifest_stats_invalid": false, 2026-02-20T18:04:42.543 INFO:tasks.workunit.client.0.trial195.stdout: "snaptrimq_len": 0, 2026-02-20T18:04:42.543 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub_duration": 0, 2026-02-20T18:04:42.543 INFO:tasks.workunit.client.0.trial195.stdout: "scrub_schedule": "no scrub is scheduled", 2026-02-20T18:04:42.543 INFO:tasks.workunit.client.0.trial195.stdout: "scrub_duration": 0, 2026-02-20T18:04:42.543 INFO:tasks.workunit.client.0.trial195.stdout: "objects_trimmed": 0, 2026-02-20T18:04:42.543 INFO:tasks.workunit.client.0.trial195.stdout: "snaptrim_duration": 0, 2026-02-20T18:04:42.543 INFO:tasks.workunit.client.0.trial195.stdout: "stat_sum": { 2026-02-20T18:04:42.543 INFO:tasks.workunit.client.0.trial195.stdout: "num_bytes": 413696, 2026-02-20T18:04:42.544 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects": 101, 2026-02-20T18:04:42.544 INFO:tasks.workunit.client.0.trial195.stdout: "num_object_clones": 0, 2026-02-20T18:04:42.544 INFO:tasks.workunit.client.0.trial195.stdout: "num_object_copies": 505, 2026-02-20T18:04:42.544 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_missing_on_primary": 0, 2026-02-20T18:04:42.544 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_missing": 1, 2026-02-20T18:04:42.544 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_degraded": 0, 2026-02-20T18:04:42.544 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_misplaced": 0, 2026-02-20T18:04:42.544 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_unfound": 0, 2026-02-20T18:04:42.544 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_dirty": 101, 2026-02-20T18:04:42.544 INFO:tasks.workunit.client.0.trial195.stdout: "num_whiteouts": 0, 2026-02-20T18:04:42.544 INFO:tasks.workunit.client.0.trial195.stdout: "num_read": 0, 2026-02-20T18:04:42.544 INFO:tasks.workunit.client.0.trial195.stdout: "num_read_kb": 0, 2026-02-20T18:04:42.544 INFO:tasks.workunit.client.0.trial195.stdout: "num_write": 101, 2026-02-20T18:04:42.544 INFO:tasks.workunit.client.0.trial195.stdout: "num_write_kb": 404, 2026-02-20T18:04:42.544 INFO:tasks.workunit.client.0.trial195.stdout: "num_scrub_errors": 0, 2026-02-20T18:04:42.544 INFO:tasks.workunit.client.0.trial195.stdout: "num_shallow_scrub_errors": 0, 2026-02-20T18:04:42.544 INFO:tasks.workunit.client.0.trial195.stdout: "num_deep_scrub_errors": 0, 2026-02-20T18:04:42.544 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_recovered": 0, 2026-02-20T18:04:42.544 INFO:tasks.workunit.client.0.trial195.stdout: "num_bytes_recovered": 0, 2026-02-20T18:04:42.545 INFO:tasks.workunit.client.0.trial195.stdout: "num_keys_recovered": 0, 2026-02-20T18:04:42.545 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_omap": 0, 2026-02-20T18:04:42.545 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_hit_set_archive": 0, 2026-02-20T18:04:42.545 INFO:tasks.workunit.client.0.trial195.stdout: "num_bytes_hit_set_archive": 0, 2026-02-20T18:04:42.545 INFO:tasks.workunit.client.0.trial195.stdout: "num_flush": 0, 2026-02-20T18:04:42.545 INFO:tasks.workunit.client.0.trial195.stdout: "num_flush_kb": 0, 2026-02-20T18:04:42.545 INFO:tasks.workunit.client.0.trial195.stdout: "num_evict": 0, 2026-02-20T18:04:42.545 INFO:tasks.workunit.client.0.trial195.stdout: "num_evict_kb": 0, 2026-02-20T18:04:42.545 INFO:tasks.workunit.client.0.trial195.stdout: "num_promote": 0, 2026-02-20T18:04:42.545 INFO:tasks.workunit.client.0.trial195.stdout: "num_flush_mode_high": 0, 2026-02-20T18:04:42.545 INFO:tasks.workunit.client.0.trial195.stdout: "num_flush_mode_low": 0, 2026-02-20T18:04:42.545 INFO:tasks.workunit.client.0.trial195.stdout: "num_evict_mode_some": 0, 2026-02-20T18:04:42.545 INFO:tasks.workunit.client.0.trial195.stdout: "num_evict_mode_full": 0, 2026-02-20T18:04:42.545 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_pinned": 0, 2026-02-20T18:04:42.545 INFO:tasks.workunit.client.0.trial195.stdout: "num_legacy_snapsets": 0, 2026-02-20T18:04:42.545 INFO:tasks.workunit.client.0.trial195.stdout: "num_large_omap_objects": 0, 2026-02-20T18:04:42.545 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_manifest": 0, 2026-02-20T18:04:42.545 INFO:tasks.workunit.client.0.trial195.stdout: "num_omap_bytes": 0, 2026-02-20T18:04:42.546 INFO:tasks.workunit.client.0.trial195.stdout: "num_omap_keys": 0, 2026-02-20T18:04:42.546 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_repaired": 0 2026-02-20T18:04:42.546 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:04:42.546 INFO:tasks.workunit.client.0.trial195.stdout: "up": [ 2026-02-20T18:04:42.546 INFO:tasks.workunit.client.0.trial195.stdout: 3, 2026-02-20T18:04:42.546 INFO:tasks.workunit.client.0.trial195.stdout: 1, 2026-02-20T18:04:42.546 INFO:tasks.workunit.client.0.trial195.stdout: 4, 2026-02-20T18:04:42.546 INFO:tasks.workunit.client.0.trial195.stdout: 0, 2026-02-20T18:04:42.546 INFO:tasks.workunit.client.0.trial195.stdout: 2 2026-02-20T18:04:42.546 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:04:42.546 INFO:tasks.workunit.client.0.trial195.stdout: "acting": [ 2026-02-20T18:04:42.546 INFO:tasks.workunit.client.0.trial195.stdout: 3, 2026-02-20T18:04:42.546 INFO:tasks.workunit.client.0.trial195.stdout: 1, 2026-02-20T18:04:42.546 INFO:tasks.workunit.client.0.trial195.stdout: 4, 2026-02-20T18:04:42.546 INFO:tasks.workunit.client.0.trial195.stdout: 0, 2026-02-20T18:04:42.546 INFO:tasks.workunit.client.0.trial195.stdout: 2 2026-02-20T18:04:42.547 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:04:42.547 INFO:tasks.workunit.client.0.trial195.stdout: "avail_no_missing": [], 2026-02-20T18:04:42.547 INFO:tasks.workunit.client.0.trial195.stdout: "object_location_counts": [], 2026-02-20T18:04:42.547 INFO:tasks.workunit.client.0.trial195.stdout: "blocked_by": [], 2026-02-20T18:04:42.547 INFO:tasks.workunit.client.0.trial195.stdout: "up_primary": 3, 2026-02-20T18:04:42.547 INFO:tasks.workunit.client.0.trial195.stdout: "acting_primary": 3, 2026-02-20T18:04:42.547 INFO:tasks.workunit.client.0.trial195.stdout: "purged_snaps": [] 2026-02-20T18:04:42.547 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:04:42.547 INFO:tasks.workunit.client.0.trial195.stdout: "empty": 0, 2026-02-20T18:04:42.547 INFO:tasks.workunit.client.0.trial195.stdout: "dne": 0, 2026-02-20T18:04:42.547 INFO:tasks.workunit.client.0.trial195.stdout: "incomplete": 0, 2026-02-20T18:04:42.547 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_started": 55, 2026-02-20T18:04:42.547 INFO:tasks.workunit.client.0.trial195.stdout: "hit_set_history": { 2026-02-20T18:04:42.547 INFO:tasks.workunit.client.0.trial195.stdout: "current_last_update": "0'0", 2026-02-20T18:04:42.547 INFO:tasks.workunit.client.0.trial195.stdout: "history": [] 2026-02-20T18:04:42.547 INFO:tasks.workunit.client.0.trial195.stdout: } 2026-02-20T18:04:42.547 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:04:42.547 INFO:tasks.workunit.client.0.trial195.stdout: { 2026-02-20T18:04:42.548 INFO:tasks.workunit.client.0.trial195.stdout: "peer": "4(2)", 2026-02-20T18:04:42.548 INFO:tasks.workunit.client.0.trial195.stdout: "pgid": "2.0s2", 2026-02-20T18:04:42.548 INFO:tasks.workunit.client.0.trial195.stdout: "shard": "2", 2026-02-20T18:04:42.548 INFO:tasks.workunit.client.0.trial195.stdout: "last_update": "48'101", 2026-02-20T18:04:42.548 INFO:tasks.workunit.client.0.trial195.stdout: "last_complete": "48'101", 2026-02-20T18:04:42.548 INFO:tasks.workunit.client.0.trial195.stdout: "log_tail": "0'0", 2026-02-20T18:04:42.548 INFO:tasks.workunit.client.0.trial195.stdout: "last_user_version": 101, 2026-02-20T18:04:42.548 INFO:tasks.workunit.client.0.trial195.stdout: "last_backfill": "MAX", 2026-02-20T18:04:42.548 INFO:tasks.workunit.client.0.trial195.stdout: "partial_writes_last_complete": [], 2026-02-20T18:04:42.548 INFO:tasks.workunit.client.0.trial195.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-20T18:04:42.548 INFO:tasks.workunit.client.0.trial195.stdout: "purged_snaps": [], 2026-02-20T18:04:42.548 INFO:tasks.workunit.client.0.trial195.stdout: "history": { 2026-02-20T18:04:42.548 INFO:tasks.workunit.client.0.trial195.stdout: "epoch_created": 41, 2026-02-20T18:04:42.548 INFO:tasks.workunit.client.0.trial195.stdout: "epoch_pool_created": 41, 2026-02-20T18:04:42.548 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_started": 55, 2026-02-20T18:04:42.548 INFO:tasks.workunit.client.0.trial195.stdout: "last_interval_started": 54, 2026-02-20T18:04:42.548 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_clean": 43, 2026-02-20T18:04:42.548 INFO:tasks.workunit.client.0.trial195.stdout: "last_interval_clean": 41, 2026-02-20T18:04:42.548 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_split": 0, 2026-02-20T18:04:42.548 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_marked_full": 0, 2026-02-20T18:04:42.549 INFO:tasks.workunit.client.0.trial195.stdout: "same_up_since": 54, 2026-02-20T18:04:42.549 INFO:tasks.workunit.client.0.trial195.stdout: "same_interval_since": 54, 2026-02-20T18:04:42.549 INFO:tasks.workunit.client.0.trial195.stdout: "same_primary_since": 41, 2026-02-20T18:04:42.549 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub": "0'0", 2026-02-20T18:04:42.549 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub_stamp": "2026-02-20T18:04:14.921735+0000", 2026-02-20T18:04:42.549 INFO:tasks.workunit.client.0.trial195.stdout: "last_deep_scrub": "0'0", 2026-02-20T18:04:42.549 INFO:tasks.workunit.client.0.trial195.stdout: "last_deep_scrub_stamp": "2026-02-20T18:04:14.921735+0000", 2026-02-20T18:04:42.549 INFO:tasks.workunit.client.0.trial195.stdout: "last_clean_scrub_stamp": "2026-02-20T18:04:14.921735+0000", 2026-02-20T18:04:42.549 INFO:tasks.workunit.client.0.trial195.stdout: "prior_readable_until_ub": 0 2026-02-20T18:04:42.549 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:04:42.549 INFO:tasks.workunit.client.0.trial195.stdout: "stats": { 2026-02-20T18:04:42.549 INFO:tasks.workunit.client.0.trial195.stdout: "version": "48'101", 2026-02-20T18:04:42.549 INFO:tasks.workunit.client.0.trial195.stdout: "reported_seq": 134, 2026-02-20T18:04:42.549 INFO:tasks.workunit.client.0.trial195.stdout: "reported_epoch": 48, 2026-02-20T18:04:42.549 INFO:tasks.workunit.client.0.trial195.stdout: "state": "active+undersized+degraded", 2026-02-20T18:04:42.549 INFO:tasks.workunit.client.0.trial195.stdout: "last_fresh": "2026-02-20T18:04:24.279491+0000", 2026-02-20T18:04:42.549 INFO:tasks.workunit.client.0.trial195.stdout: "last_change": "2026-02-20T18:04:21.673339+0000", 2026-02-20T18:04:42.549 INFO:tasks.workunit.client.0.trial195.stdout: "last_active": "2026-02-20T18:04:24.279491+0000", 2026-02-20T18:04:42.549 INFO:tasks.workunit.client.0.trial195.stdout: "last_peered": "2026-02-20T18:04:24.279491+0000", 2026-02-20T18:04:42.550 INFO:tasks.workunit.client.0.trial195.stdout: "last_clean": "2026-02-20T18:04:21.320608+0000", 2026-02-20T18:04:42.550 INFO:tasks.workunit.client.0.trial195.stdout: "last_became_active": "2026-02-20T18:04:21.673339+0000", 2026-02-20T18:04:42.550 INFO:tasks.workunit.client.0.trial195.stdout: "last_became_peered": "2026-02-20T18:04:21.673339+0000", 2026-02-20T18:04:42.550 INFO:tasks.workunit.client.0.trial195.stdout: "last_unstale": "2026-02-20T18:04:24.279491+0000", 2026-02-20T18:04:42.550 INFO:tasks.workunit.client.0.trial195.stdout: "last_undegraded": "2026-02-20T18:04:21.672159+0000", 2026-02-20T18:04:42.550 INFO:tasks.workunit.client.0.trial195.stdout: "last_fullsized": "2026-02-20T18:04:21.672055+0000", 2026-02-20T18:04:42.550 INFO:tasks.workunit.client.0.trial195.stdout: "mapping_epoch": 54, 2026-02-20T18:04:42.550 INFO:tasks.workunit.client.0.trial195.stdout: "log_start": "0'0", 2026-02-20T18:04:42.550 INFO:tasks.workunit.client.0.trial195.stdout: "ondisk_log_start": "0'0", 2026-02-20T18:04:42.550 INFO:tasks.workunit.client.0.trial195.stdout: "created": 41, 2026-02-20T18:04:42.550 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_clean": 43, 2026-02-20T18:04:42.550 INFO:tasks.workunit.client.0.trial195.stdout: "parent": "0.0", 2026-02-20T18:04:42.550 INFO:tasks.workunit.client.0.trial195.stdout: "parent_split_bits": 0, 2026-02-20T18:04:42.550 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub": "0'0", 2026-02-20T18:04:42.550 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub_stamp": "2026-02-20T18:04:14.921735+0000", 2026-02-20T18:04:42.550 INFO:tasks.workunit.client.0.trial195.stdout: "last_deep_scrub": "0'0", 2026-02-20T18:04:42.550 INFO:tasks.workunit.client.0.trial195.stdout: "last_deep_scrub_stamp": "2026-02-20T18:04:14.921735+0000", 2026-02-20T18:04:42.550 INFO:tasks.workunit.client.0.trial195.stdout: "last_clean_scrub_stamp": "2026-02-20T18:04:14.921735+0000", 2026-02-20T18:04:42.550 INFO:tasks.workunit.client.0.trial195.stdout: "objects_scrubbed": 0, 2026-02-20T18:04:42.550 INFO:tasks.workunit.client.0.trial195.stdout: "log_size": 101, 2026-02-20T18:04:42.550 INFO:tasks.workunit.client.0.trial195.stdout: "log_dups_size": 0, 2026-02-20T18:04:42.550 INFO:tasks.workunit.client.0.trial195.stdout: "ondisk_log_size": 101, 2026-02-20T18:04:42.551 INFO:tasks.workunit.client.0.trial195.stdout: "stats_invalid": false, 2026-02-20T18:04:42.551 INFO:tasks.workunit.client.0.trial195.stdout: "dirty_stats_invalid": false, 2026-02-20T18:04:42.551 INFO:tasks.workunit.client.0.trial195.stdout: "omap_stats_invalid": false, 2026-02-20T18:04:42.551 INFO:tasks.workunit.client.0.trial195.stdout: "hitset_stats_invalid": false, 2026-02-20T18:04:42.551 INFO:tasks.workunit.client.0.trial195.stdout: "hitset_bytes_stats_invalid": false, 2026-02-20T18:04:42.551 INFO:tasks.workunit.client.0.trial195.stdout: "pin_stats_invalid": false, 2026-02-20T18:04:42.551 INFO:tasks.workunit.client.0.trial195.stdout: "manifest_stats_invalid": false, 2026-02-20T18:04:42.551 INFO:tasks.workunit.client.0.trial195.stdout: "snaptrimq_len": 0, 2026-02-20T18:04:42.551 INFO:tasks.workunit.client.0.trial195.stdout: "last_scrub_duration": 0, 2026-02-20T18:04:42.551 INFO:tasks.workunit.client.0.trial195.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-21T20:50:09.597014+0000", 2026-02-20T18:04:42.551 INFO:tasks.workunit.client.0.trial195.stdout: "scrub_duration": 0, 2026-02-20T18:04:42.551 INFO:tasks.workunit.client.0.trial195.stdout: "objects_trimmed": 0, 2026-02-20T18:04:42.551 INFO:tasks.workunit.client.0.trial195.stdout: "snaptrim_duration": 0, 2026-02-20T18:04:42.551 INFO:tasks.workunit.client.0.trial195.stdout: "stat_sum": { 2026-02-20T18:04:42.551 INFO:tasks.workunit.client.0.trial195.stdout: "num_bytes": 413696, 2026-02-20T18:04:42.551 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects": 101, 2026-02-20T18:04:42.551 INFO:tasks.workunit.client.0.trial195.stdout: "num_object_clones": 0, 2026-02-20T18:04:42.551 INFO:tasks.workunit.client.0.trial195.stdout: "num_object_copies": 505, 2026-02-20T18:04:42.551 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_missing_on_primary": 0, 2026-02-20T18:04:42.551 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_missing": 0, 2026-02-20T18:04:42.551 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_degraded": 101, 2026-02-20T18:04:42.552 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_misplaced": 0, 2026-02-20T18:04:42.552 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_unfound": 0, 2026-02-20T18:04:42.552 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_dirty": 101, 2026-02-20T18:04:42.552 INFO:tasks.workunit.client.0.trial195.stdout: "num_whiteouts": 0, 2026-02-20T18:04:42.552 INFO:tasks.workunit.client.0.trial195.stdout: "num_read": 0, 2026-02-20T18:04:42.552 INFO:tasks.workunit.client.0.trial195.stdout: "num_read_kb": 0, 2026-02-20T18:04:42.552 INFO:tasks.workunit.client.0.trial195.stdout: "num_write": 101, 2026-02-20T18:04:42.552 INFO:tasks.workunit.client.0.trial195.stdout: "num_write_kb": 404, 2026-02-20T18:04:42.552 INFO:tasks.workunit.client.0.trial195.stdout: "num_scrub_errors": 0, 2026-02-20T18:04:42.552 INFO:tasks.workunit.client.0.trial195.stdout: "num_shallow_scrub_errors": 0, 2026-02-20T18:04:42.552 INFO:tasks.workunit.client.0.trial195.stdout: "num_deep_scrub_errors": 0, 2026-02-20T18:04:42.552 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_recovered": 0, 2026-02-20T18:04:42.552 INFO:tasks.workunit.client.0.trial195.stdout: "num_bytes_recovered": 0, 2026-02-20T18:04:42.552 INFO:tasks.workunit.client.0.trial195.stdout: "num_keys_recovered": 0, 2026-02-20T18:04:42.552 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_omap": 0, 2026-02-20T18:04:42.552 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_hit_set_archive": 0, 2026-02-20T18:04:42.552 INFO:tasks.workunit.client.0.trial195.stdout: "num_bytes_hit_set_archive": 0, 2026-02-20T18:04:42.552 INFO:tasks.workunit.client.0.trial195.stdout: "num_flush": 0, 2026-02-20T18:04:42.552 INFO:tasks.workunit.client.0.trial195.stdout: "num_flush_kb": 0, 2026-02-20T18:04:42.552 INFO:tasks.workunit.client.0.trial195.stdout: "num_evict": 0, 2026-02-20T18:04:42.552 INFO:tasks.workunit.client.0.trial195.stdout: "num_evict_kb": 0, 2026-02-20T18:04:42.552 INFO:tasks.workunit.client.0.trial195.stdout: "num_promote": 0, 2026-02-20T18:04:42.553 INFO:tasks.workunit.client.0.trial195.stdout: "num_flush_mode_high": 0, 2026-02-20T18:04:42.553 INFO:tasks.workunit.client.0.trial195.stdout: "num_flush_mode_low": 0, 2026-02-20T18:04:42.553 INFO:tasks.workunit.client.0.trial195.stdout: "num_evict_mode_some": 0, 2026-02-20T18:04:42.553 INFO:tasks.workunit.client.0.trial195.stdout: "num_evict_mode_full": 0, 2026-02-20T18:04:42.553 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_pinned": 0, 2026-02-20T18:04:42.553 INFO:tasks.workunit.client.0.trial195.stdout: "num_legacy_snapsets": 0, 2026-02-20T18:04:42.553 INFO:tasks.workunit.client.0.trial195.stdout: "num_large_omap_objects": 0, 2026-02-20T18:04:42.553 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_manifest": 0, 2026-02-20T18:04:42.553 INFO:tasks.workunit.client.0.trial195.stdout: "num_omap_bytes": 0, 2026-02-20T18:04:42.553 INFO:tasks.workunit.client.0.trial195.stdout: "num_omap_keys": 0, 2026-02-20T18:04:42.553 INFO:tasks.workunit.client.0.trial195.stdout: "num_objects_repaired": 0 2026-02-20T18:04:42.553 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:04:42.553 INFO:tasks.workunit.client.0.trial195.stdout: "up": [ 2026-02-20T18:04:42.553 INFO:tasks.workunit.client.0.trial195.stdout: 3, 2026-02-20T18:04:42.553 INFO:tasks.workunit.client.0.trial195.stdout: 1, 2026-02-20T18:04:42.553 INFO:tasks.workunit.client.0.trial195.stdout: 4, 2026-02-20T18:04:42.553 INFO:tasks.workunit.client.0.trial195.stdout: 0, 2026-02-20T18:04:42.553 INFO:tasks.workunit.client.0.trial195.stdout: 2 2026-02-20T18:04:42.553 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:04:42.553 INFO:tasks.workunit.client.0.trial195.stdout: "acting": [ 2026-02-20T18:04:42.553 INFO:tasks.workunit.client.0.trial195.stdout: 3, 2026-02-20T18:04:42.553 INFO:tasks.workunit.client.0.trial195.stdout: 1, 2026-02-20T18:04:42.553 INFO:tasks.workunit.client.0.trial195.stdout: 4, 2026-02-20T18:04:42.553 INFO:tasks.workunit.client.0.trial195.stdout: 0, 2026-02-20T18:04:42.554 INFO:tasks.workunit.client.0.trial195.stdout: 2 2026-02-20T18:04:42.554 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:04:42.554 INFO:tasks.workunit.client.0.trial195.stdout: "avail_no_missing": [ 2026-02-20T18:04:42.554 INFO:tasks.workunit.client.0.trial195.stdout: "3(0)", 2026-02-20T18:04:42.554 INFO:tasks.workunit.client.0.trial195.stdout: "0(3)", 2026-02-20T18:04:42.554 INFO:tasks.workunit.client.0.trial195.stdout: "1(1)", 2026-02-20T18:04:42.554 INFO:tasks.workunit.client.0.trial195.stdout: "4(2)" 2026-02-20T18:04:42.554 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:04:42.554 INFO:tasks.workunit.client.0.trial195.stdout: "object_location_counts": [ 2026-02-20T18:04:42.554 INFO:tasks.workunit.client.0.trial195.stdout: { 2026-02-20T18:04:42.554 INFO:tasks.workunit.client.0.trial195.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-20T18:04:42.554 INFO:tasks.workunit.client.0.trial195.stdout: "objects": 101 2026-02-20T18:04:42.554 INFO:tasks.workunit.client.0.trial195.stdout: } 2026-02-20T18:04:42.554 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:04:42.554 INFO:tasks.workunit.client.0.trial195.stdout: "blocked_by": [], 2026-02-20T18:04:42.554 INFO:tasks.workunit.client.0.trial195.stdout: "up_primary": 3, 2026-02-20T18:04:42.554 INFO:tasks.workunit.client.0.trial195.stdout: "acting_primary": 3, 2026-02-20T18:04:42.554 INFO:tasks.workunit.client.0.trial195.stdout: "purged_snaps": [] 2026-02-20T18:04:42.554 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:04:42.554 INFO:tasks.workunit.client.0.trial195.stdout: "empty": 0, 2026-02-20T18:04:42.554 INFO:tasks.workunit.client.0.trial195.stdout: "dne": 0, 2026-02-20T18:04:42.554 INFO:tasks.workunit.client.0.trial195.stdout: "incomplete": 0, 2026-02-20T18:04:42.554 INFO:tasks.workunit.client.0.trial195.stdout: "last_epoch_started": 55, 2026-02-20T18:04:42.555 INFO:tasks.workunit.client.0.trial195.stdout: "hit_set_history": { 2026-02-20T18:04:42.555 INFO:tasks.workunit.client.0.trial195.stdout: "current_last_update": "0'0", 2026-02-20T18:04:42.555 INFO:tasks.workunit.client.0.trial195.stdout: "history": [] 2026-02-20T18:04:42.555 INFO:tasks.workunit.client.0.trial195.stdout: } 2026-02-20T18:04:42.555 INFO:tasks.workunit.client.0.trial195.stdout: } 2026-02-20T18:04:42.555 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:04:42.555 INFO:tasks.workunit.client.0.trial195.stdout: "recovery_state": [ 2026-02-20T18:04:42.555 INFO:tasks.workunit.client.0.trial195.stdout: { 2026-02-20T18:04:42.555 INFO:tasks.workunit.client.0.trial195.stdout: "name": "Started/Primary/Active", 2026-02-20T18:04:42.555 INFO:tasks.workunit.client.0.trial195.stdout: "enter_time": "2026-02-20T18:04:26.949487+0000", 2026-02-20T18:04:42.555 INFO:tasks.workunit.client.0.trial195.stdout: "might_have_unfound": [ 2026-02-20T18:04:42.555 INFO:tasks.workunit.client.0.trial195.stdout: { 2026-02-20T18:04:42.555 INFO:tasks.workunit.client.0.trial195.stdout: "osd": "0(3)", 2026-02-20T18:04:42.555 INFO:tasks.workunit.client.0.trial195.stdout: "status": "already probed" 2026-02-20T18:04:42.555 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:04:42.555 INFO:tasks.workunit.client.0.trial195.stdout: { 2026-02-20T18:04:42.555 INFO:tasks.workunit.client.0.trial195.stdout: "osd": "1(1)", 2026-02-20T18:04:42.555 INFO:tasks.workunit.client.0.trial195.stdout: "status": "already probed" 2026-02-20T18:04:42.555 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:04:42.555 INFO:tasks.workunit.client.0.trial195.stdout: { 2026-02-20T18:04:42.555 INFO:tasks.workunit.client.0.trial195.stdout: "osd": "2(4)", 2026-02-20T18:04:42.555 INFO:tasks.workunit.client.0.trial195.stdout: "status": "already probed" 2026-02-20T18:04:42.556 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:04:42.556 INFO:tasks.workunit.client.0.trial195.stdout: { 2026-02-20T18:04:42.556 INFO:tasks.workunit.client.0.trial195.stdout: "osd": "4(2)", 2026-02-20T18:04:42.556 INFO:tasks.workunit.client.0.trial195.stdout: "status": "already probed" 2026-02-20T18:04:42.556 INFO:tasks.workunit.client.0.trial195.stdout: } 2026-02-20T18:04:42.556 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:04:42.556 INFO:tasks.workunit.client.0.trial195.stdout: "recovery_progress": { 2026-02-20T18:04:42.556 INFO:tasks.workunit.client.0.trial195.stdout: "backfill_targets": [], 2026-02-20T18:04:42.556 INFO:tasks.workunit.client.0.trial195.stdout: "waiting_on_backfill": [], 2026-02-20T18:04:42.556 INFO:tasks.workunit.client.0.trial195.stdout: "last_backfill_started": "MIN", 2026-02-20T18:04:42.556 INFO:tasks.workunit.client.0.trial195.stdout: "backfill_info": { 2026-02-20T18:04:42.556 INFO:tasks.workunit.client.0.trial195.stdout: "begin": "MIN", 2026-02-20T18:04:42.556 INFO:tasks.workunit.client.0.trial195.stdout: "end": "MIN", 2026-02-20T18:04:42.556 INFO:tasks.workunit.client.0.trial195.stdout: "objects": [] 2026-02-20T18:04:42.556 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:04:42.556 INFO:tasks.workunit.client.0.trial195.stdout: "peer_backfill_info": [], 2026-02-20T18:04:42.556 INFO:tasks.workunit.client.0.trial195.stdout: "backfills_in_flight": [], 2026-02-20T18:04:42.556 INFO:tasks.workunit.client.0.trial195.stdout: "recovering": [], 2026-02-20T18:04:42.556 INFO:tasks.workunit.client.0.trial195.stdout: "pg_backend": { 2026-02-20T18:04:42.556 INFO:tasks.workunit.client.0.trial195.stdout: "recovery_ops": [], 2026-02-20T18:04:42.556 INFO:tasks.workunit.client.0.trial195.stdout: "read_ops": [] 2026-02-20T18:04:42.556 INFO:tasks.workunit.client.0.trial195.stdout: } 2026-02-20T18:04:42.556 INFO:tasks.workunit.client.0.trial195.stdout: } 2026-02-20T18:04:42.557 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:04:42.557 INFO:tasks.workunit.client.0.trial195.stdout: { 2026-02-20T18:04:42.557 INFO:tasks.workunit.client.0.trial195.stdout: "name": "Started", 2026-02-20T18:04:42.557 INFO:tasks.workunit.client.0.trial195.stdout: "enter_time": "2026-02-20T18:04:26.847669+0000" 2026-02-20T18:04:42.557 INFO:tasks.workunit.client.0.trial195.stdout: } 2026-02-20T18:04:42.557 INFO:tasks.workunit.client.0.trial195.stdout: ], 2026-02-20T18:04:42.557 INFO:tasks.workunit.client.0.trial195.stdout: "scrubber": { 2026-02-20T18:04:42.557 INFO:tasks.workunit.client.0.trial195.stdout: "active": false, 2026-02-20T18:04:42.557 INFO:tasks.workunit.client.0.trial195.stdout: "must_scrub": false, 2026-02-20T18:04:42.557 INFO:tasks.workunit.client.0.trial195.stdout: "must_deep_scrub": false, 2026-02-20T18:04:42.557 INFO:tasks.workunit.client.0.trial195.stdout: "must_repair": false, 2026-02-20T18:04:42.557 INFO:tasks.workunit.client.0.trial195.stdout: "scrub_reg_stamp": "2026-02-21T20:50:09.597014+0000", 2026-02-20T18:04:42.557 INFO:tasks.workunit.client.0.trial195.stdout: "schedule": "not registered for scrubbing" 2026-02-20T18:04:42.557 INFO:tasks.workunit.client.0.trial195.stdout: }, 2026-02-20T18:04:42.557 INFO:tasks.workunit.client.0.trial195.stdout: "agent_state": {} 2026-02-20T18:04:42.557 INFO:tasks.workunit.client.0.trial195.stdout:} 2026-02-20T18:04:42.558 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:684: TEST_ec_recovery_unfound: ceph pg 2.0 list_unfound 2026-02-20T18:04:42.558 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:684: TEST_ec_recovery_unfound: grep -q obj75 2026-02-20T18:04:42.570 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: ceph pg 2.0 list_unfound 2026-02-20T18:04:42.570 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: jq .available_might_have_unfound 2026-02-20T18:04:42.638 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: check=true 2026-02-20T18:04:42.639 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: test true == true 2026-02-20T18:04:42.639 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:689: TEST_ec_recovery_unfound: ceph pg 2.0 list_unfound 2026-02-20T18:04:42.639 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:689: TEST_ec_recovery_unfound: jq '.might_have_unfound | length' 2026-02-20T18:04:42.705 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:689: TEST_ec_recovery_unfound: check=0 2026-02-20T18:04:42.705 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:690: TEST_ec_recovery_unfound: test 0 == 0 2026-02-20T18:04:42.705 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:693: TEST_ec_recovery_unfound: timeout 5 rados -p pool-jerasure get obj75 td/test-erasure-eio/CHECK 2026-02-20T18:04:47.709 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:694: TEST_ec_recovery_unfound: test 124 = 124 2026-02-20T18:04:47.710 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:696: TEST_ec_recovery_unfound: ceph pg 2.0 mark_unfound_lost delete 2026-02-20T18:04:47.776 INFO:tasks.workunit.client.0.trial195.stderr:pg has no unfound objects 2026-02-20T18:04:47.781 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:698: TEST_ec_recovery_unfound: wait_for_clean 2026-02-20T18:04:47.781 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:04:47.781 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:04:47.781 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:04:47.781 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:04:47.782 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:04:47.782 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:04:47.782 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:04:47.782 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:04:47.782 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:04:47.869 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:04:47.869 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:04:47.869 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:04:47.869 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:04:47.869 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:04:47.869 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:04:48.092 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:04:48.092 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:04:48.092 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:04:48.092 INFO:tasks.workunit.client.0.trial195.stderr:3 2026-02-20T18:04:48.093 INFO:tasks.workunit.client.0.trial195.stderr:4' 2026-02-20T18:04:48.093 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:04:48.093 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:04:48.093 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:04:48.159 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803789 2026-02-20T18:04:48.160 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803789 2026-02-20T18:04:48.160 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803789' 2026-02-20T18:04:48.160 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:04:48.160 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:04:48.225 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574861 2026-02-20T18:04:48.225 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574861 2026-02-20T18:04:48.225 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803789 1-55834574861' 2026-02-20T18:04:48.225 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:04:48.225 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:04:48.291 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=214748364807 2026-02-20T18:04:48.292 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 214748364807 2026-02-20T18:04:48.292 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803789 1-55834574861 2-214748364807' 2026-02-20T18:04:48.292 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:04:48.292 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:04:48.360 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149708 2026-02-20T18:04:48.360 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149708 2026-02-20T18:04:48.360 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803789 1-55834574861 2-214748364807 3-111669149708' 2026-02-20T18:04:48.360 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:04:48.361 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-20T18:04:48.424 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=154618822667 2026-02-20T18:04:48.424 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 154618822667 2026-02-20T18:04:48.424 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803789 1-55834574861 2-214748364807 3-111669149708 4-154618822667' 2026-02-20T18:04:48.424 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:04:48.425 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803789 2026-02-20T18:04:48.425 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:04:48.426 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:04:48.426 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803789 2026-02-20T18:04:48.426 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:04:48.427 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803789 2026-02-20T18:04:48.428 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.0 seq 25769803789 2026-02-20T18:04:48.428 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803789' 2026-02-20T18:04:48.428 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:04:48.653 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803789 -lt 25769803789 2026-02-20T18:04:48.653 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:04:48.653 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574861 2026-02-20T18:04:48.653 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:04:48.654 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:04:48.655 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-55834574861 2026-02-20T18:04:48.655 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:04:48.656 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574861 2026-02-20T18:04:48.656 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 55834574861' 2026-02-20T18:04:48.656 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.1 seq 55834574861 2026-02-20T18:04:48.656 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:04:48.885 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574861 -lt 55834574861 2026-02-20T18:04:48.885 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:04:48.886 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-214748364807 2026-02-20T18:04:48.886 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:04:48.887 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:04:48.888 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-214748364807 2026-02-20T18:04:48.888 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:04:48.889 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=214748364807 2026-02-20T18:04:48.889 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.2 seq 214748364807 2026-02-20T18:04:48.890 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 214748364807' 2026-02-20T18:04:48.890 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:04:49.114 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 214748364807 -lt 214748364807 2026-02-20T18:04:49.114 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:04:49.115 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149708 2026-02-20T18:04:49.115 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:04:49.116 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:04:49.116 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149708 2026-02-20T18:04:49.116 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:04:49.118 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149708 2026-02-20T18:04:49.118 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.3 seq 111669149708 2026-02-20T18:04:49.118 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149708' 2026-02-20T18:04:49.118 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:04:49.349 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149708 -lt 111669149708 2026-02-20T18:04:49.350 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:04:49.350 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-154618822667 2026-02-20T18:04:49.350 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:04:49.350 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-20T18:04:49.351 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-154618822667 2026-02-20T18:04:49.351 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:04:49.352 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=154618822667 2026-02-20T18:04:49.352 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 154618822667' 2026-02-20T18:04:49.352 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.4 seq 154618822667 2026-02-20T18:04:49.352 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-20T18:04:49.574 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 154618822665 -lt 154618822667 2026-02-20T18:04:49.574 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:04:50.575 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:04:50.575 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-20T18:04:50.801 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 154618822668 -lt 154618822667 2026-02-20T18:04:50.802 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:04:50.802 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:04:50.802 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:04:51.131 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:04:51.131 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:04:51.131 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:04:51.131 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:04:51.131 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:04:51.131 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:04:51.131 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:04:51.131 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:04:51.352 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:04:51.352 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:04:51.353 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:04:51.353 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:04:51.666 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:04:51.667 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:04:51.667 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:04:51.667 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: seq 1 100 2026-02-20T18:04:51.667 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:51.668 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj1 = obj75 ']' 2026-02-20T18:04:51.668 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj1 td/test-erasure-eio/CHECK 2026-02-20T18:04:51.687 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:51.688 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:51.688 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj2 = obj75 ']' 2026-02-20T18:04:51.688 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj2 td/test-erasure-eio/CHECK 2026-02-20T18:04:51.707 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:51.708 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:51.708 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj3 = obj75 ']' 2026-02-20T18:04:51.708 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj3 td/test-erasure-eio/CHECK 2026-02-20T18:04:51.726 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:51.728 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:51.728 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj4 = obj75 ']' 2026-02-20T18:04:51.728 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj4 td/test-erasure-eio/CHECK 2026-02-20T18:04:51.746 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:51.747 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:51.747 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj5 = obj75 ']' 2026-02-20T18:04:51.747 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj5 td/test-erasure-eio/CHECK 2026-02-20T18:04:51.766 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:51.767 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:51.767 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj6 = obj75 ']' 2026-02-20T18:04:51.768 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj6 td/test-erasure-eio/CHECK 2026-02-20T18:04:51.786 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:51.787 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:51.787 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj7 = obj75 ']' 2026-02-20T18:04:51.787 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj7 td/test-erasure-eio/CHECK 2026-02-20T18:04:51.805 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:51.806 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:51.806 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj8 = obj75 ']' 2026-02-20T18:04:51.806 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj8 td/test-erasure-eio/CHECK 2026-02-20T18:04:51.825 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:51.826 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:51.826 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj9 = obj75 ']' 2026-02-20T18:04:51.826 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj9 td/test-erasure-eio/CHECK 2026-02-20T18:04:51.844 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:51.845 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:51.845 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj10 = obj75 ']' 2026-02-20T18:04:51.845 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj10 td/test-erasure-eio/CHECK 2026-02-20T18:04:51.862 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:51.863 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:51.863 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj11 = obj75 ']' 2026-02-20T18:04:51.864 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj11 td/test-erasure-eio/CHECK 2026-02-20T18:04:51.882 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:51.883 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:51.883 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj12 = obj75 ']' 2026-02-20T18:04:51.883 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj12 td/test-erasure-eio/CHECK 2026-02-20T18:04:51.901 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:51.902 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:51.902 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj13 = obj75 ']' 2026-02-20T18:04:51.902 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj13 td/test-erasure-eio/CHECK 2026-02-20T18:04:51.920 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:51.921 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:51.921 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj14 = obj75 ']' 2026-02-20T18:04:51.921 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj14 td/test-erasure-eio/CHECK 2026-02-20T18:04:51.939 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:51.939 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:51.939 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj15 = obj75 ']' 2026-02-20T18:04:51.939 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj15 td/test-erasure-eio/CHECK 2026-02-20T18:04:51.958 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:51.959 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:51.959 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj16 = obj75 ']' 2026-02-20T18:04:51.959 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj16 td/test-erasure-eio/CHECK 2026-02-20T18:04:51.977 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:51.978 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:51.978 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj17 = obj75 ']' 2026-02-20T18:04:51.978 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj17 td/test-erasure-eio/CHECK 2026-02-20T18:04:51.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:51.997 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:51.998 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj18 = obj75 ']' 2026-02-20T18:04:51.998 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj18 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.016 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.017 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.017 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj19 = obj75 ']' 2026-02-20T18:04:52.017 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj19 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.035 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj20 = obj75 ']' 2026-02-20T18:04:52.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj20 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.054 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.056 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.056 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj21 = obj75 ']' 2026-02-20T18:04:52.056 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj21 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.073 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.074 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.074 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj22 = obj75 ']' 2026-02-20T18:04:52.074 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj22 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.092 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.093 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.094 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj23 = obj75 ']' 2026-02-20T18:04:52.094 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj23 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.112 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.113 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.113 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj24 = obj75 ']' 2026-02-20T18:04:52.113 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj24 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.132 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.133 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.134 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj25 = obj75 ']' 2026-02-20T18:04:52.134 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj25 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.151 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.152 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.152 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj26 = obj75 ']' 2026-02-20T18:04:52.152 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj26 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.170 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.171 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.171 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj27 = obj75 ']' 2026-02-20T18:04:52.172 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj27 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.190 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj28 = obj75 ']' 2026-02-20T18:04:52.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj28 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.209 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.210 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.210 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj29 = obj75 ']' 2026-02-20T18:04:52.210 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj29 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.228 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.229 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.229 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj30 = obj75 ']' 2026-02-20T18:04:52.229 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj30 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.247 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.248 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.248 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj31 = obj75 ']' 2026-02-20T18:04:52.248 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj31 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.265 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.266 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.266 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj32 = obj75 ']' 2026-02-20T18:04:52.267 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj32 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.285 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.286 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.286 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj33 = obj75 ']' 2026-02-20T18:04:52.286 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj33 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.304 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.305 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.305 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj34 = obj75 ']' 2026-02-20T18:04:52.305 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj34 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.323 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.324 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.324 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj35 = obj75 ']' 2026-02-20T18:04:52.324 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj35 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.342 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.343 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.343 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj36 = obj75 ']' 2026-02-20T18:04:52.343 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj36 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.362 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.362 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.363 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj37 = obj75 ']' 2026-02-20T18:04:52.363 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj37 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.380 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.381 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.381 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj38 = obj75 ']' 2026-02-20T18:04:52.381 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj38 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.399 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.400 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.400 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj39 = obj75 ']' 2026-02-20T18:04:52.400 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj39 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.419 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.420 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.420 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj40 = obj75 ']' 2026-02-20T18:04:52.420 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj40 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.439 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.439 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.440 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj41 = obj75 ']' 2026-02-20T18:04:52.440 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj41 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.458 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.459 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.459 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj42 = obj75 ']' 2026-02-20T18:04:52.459 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj42 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.477 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.478 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.479 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj43 = obj75 ']' 2026-02-20T18:04:52.479 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj43 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.496 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.497 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.497 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj44 = obj75 ']' 2026-02-20T18:04:52.497 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj44 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.515 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.516 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.516 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj45 = obj75 ']' 2026-02-20T18:04:52.516 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj45 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.534 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.535 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.535 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj46 = obj75 ']' 2026-02-20T18:04:52.535 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj46 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.553 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj47 = obj75 ']' 2026-02-20T18:04:52.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj47 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.572 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.573 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.573 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj48 = obj75 ']' 2026-02-20T18:04:52.573 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj48 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.592 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.592 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.592 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj49 = obj75 ']' 2026-02-20T18:04:52.592 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj49 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.611 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.612 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.612 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj50 = obj75 ']' 2026-02-20T18:04:52.612 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj50 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.630 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.631 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.631 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj51 = obj75 ']' 2026-02-20T18:04:52.631 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj51 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.649 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.650 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.650 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj52 = obj75 ']' 2026-02-20T18:04:52.651 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj52 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj53 = obj75 ']' 2026-02-20T18:04:52.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj53 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.689 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.690 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.690 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj54 = obj75 ']' 2026-02-20T18:04:52.690 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj54 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.707 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.708 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.709 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj55 = obj75 ']' 2026-02-20T18:04:52.709 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj55 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.726 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.727 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.727 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj56 = obj75 ']' 2026-02-20T18:04:52.728 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj56 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.746 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.747 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.747 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj57 = obj75 ']' 2026-02-20T18:04:52.747 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj57 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.767 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.768 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.768 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj58 = obj75 ']' 2026-02-20T18:04:52.768 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj58 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.786 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.787 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.787 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj59 = obj75 ']' 2026-02-20T18:04:52.787 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj59 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.807 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.808 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.808 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj60 = obj75 ']' 2026-02-20T18:04:52.808 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj60 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.826 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.827 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.827 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj61 = obj75 ']' 2026-02-20T18:04:52.827 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj61 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.845 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.846 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.846 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj62 = obj75 ']' 2026-02-20T18:04:52.846 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj62 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.865 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.866 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.866 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj63 = obj75 ']' 2026-02-20T18:04:52.866 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj63 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.884 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.885 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.885 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj64 = obj75 ']' 2026-02-20T18:04:52.885 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj64 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.904 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.905 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.905 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj65 = obj75 ']' 2026-02-20T18:04:52.905 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj65 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.924 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.924 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.925 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj66 = obj75 ']' 2026-02-20T18:04:52.925 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj66 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.942 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj67 = obj75 ']' 2026-02-20T18:04:52.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj67 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.961 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.962 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.962 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj68 = obj75 ']' 2026-02-20T18:04:52.962 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj68 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.980 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:52.981 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:52.981 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj69 = obj75 ']' 2026-02-20T18:04:52.981 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj69 td/test-erasure-eio/CHECK 2026-02-20T18:04:52.999 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:53.000 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:53.000 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj70 = obj75 ']' 2026-02-20T18:04:53.000 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj70 td/test-erasure-eio/CHECK 2026-02-20T18:04:53.017 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:53.018 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:53.018 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj71 = obj75 ']' 2026-02-20T18:04:53.019 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj71 td/test-erasure-eio/CHECK 2026-02-20T18:04:53.037 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:53.038 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:53.038 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj72 = obj75 ']' 2026-02-20T18:04:53.038 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj72 td/test-erasure-eio/CHECK 2026-02-20T18:04:53.056 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:53.057 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:53.057 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj73 = obj75 ']' 2026-02-20T18:04:53.057 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj73 td/test-erasure-eio/CHECK 2026-02-20T18:04:53.075 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:53.076 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:53.076 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj74 = obj75 ']' 2026-02-20T18:04:53.076 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj74 td/test-erasure-eio/CHECK 2026-02-20T18:04:53.094 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:53.095 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:53.095 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj75 = obj75 ']' 2026-02-20T18:04:53.095 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:704: TEST_ec_recovery_unfound: rados -p pool-jerasure get obj75 td/test-erasure-eio/CHECK 2026-02-20T18:04:53.111 INFO:tasks.workunit.client.0.trial195.stderr:error getting pool-jerasure/obj75: (2) No such file or directory 2026-02-20T18:04:53.112 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:53.113 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj76 = obj75 ']' 2026-02-20T18:04:53.113 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj76 td/test-erasure-eio/CHECK 2026-02-20T18:04:53.131 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:53.132 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:53.132 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj77 = obj75 ']' 2026-02-20T18:04:53.132 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj77 td/test-erasure-eio/CHECK 2026-02-20T18:04:53.151 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:53.152 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:53.152 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj78 = obj75 ']' 2026-02-20T18:04:53.152 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj78 td/test-erasure-eio/CHECK 2026-02-20T18:04:53.169 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:53.171 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:53.171 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj79 = obj75 ']' 2026-02-20T18:04:53.171 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj79 td/test-erasure-eio/CHECK 2026-02-20T18:04:53.188 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:53.189 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:53.189 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj80 = obj75 ']' 2026-02-20T18:04:53.189 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj80 td/test-erasure-eio/CHECK 2026-02-20T18:04:53.207 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:53.208 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:53.208 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj81 = obj75 ']' 2026-02-20T18:04:53.208 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj81 td/test-erasure-eio/CHECK 2026-02-20T18:04:53.226 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:53.227 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:53.227 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj82 = obj75 ']' 2026-02-20T18:04:53.228 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj82 td/test-erasure-eio/CHECK 2026-02-20T18:04:53.245 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:53.246 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:53.246 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj83 = obj75 ']' 2026-02-20T18:04:53.246 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj83 td/test-erasure-eio/CHECK 2026-02-20T18:04:53.263 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:53.264 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:53.264 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj84 = obj75 ']' 2026-02-20T18:04:53.265 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj84 td/test-erasure-eio/CHECK 2026-02-20T18:04:53.283 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:53.284 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:53.284 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj85 = obj75 ']' 2026-02-20T18:04:53.284 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj85 td/test-erasure-eio/CHECK 2026-02-20T18:04:53.302 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:53.303 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:53.303 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj86 = obj75 ']' 2026-02-20T18:04:53.303 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj86 td/test-erasure-eio/CHECK 2026-02-20T18:04:53.321 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:53.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:53.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj87 = obj75 ']' 2026-02-20T18:04:53.323 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj87 td/test-erasure-eio/CHECK 2026-02-20T18:04:53.340 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:53.341 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:53.342 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj88 = obj75 ']' 2026-02-20T18:04:53.342 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj88 td/test-erasure-eio/CHECK 2026-02-20T18:04:53.360 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:53.361 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:53.361 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj89 = obj75 ']' 2026-02-20T18:04:53.361 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj89 td/test-erasure-eio/CHECK 2026-02-20T18:04:53.379 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:53.380 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:53.380 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj90 = obj75 ']' 2026-02-20T18:04:53.380 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj90 td/test-erasure-eio/CHECK 2026-02-20T18:04:53.399 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:53.400 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:53.400 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj91 = obj75 ']' 2026-02-20T18:04:53.400 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj91 td/test-erasure-eio/CHECK 2026-02-20T18:04:53.417 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:53.418 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:53.419 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj92 = obj75 ']' 2026-02-20T18:04:53.419 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj92 td/test-erasure-eio/CHECK 2026-02-20T18:04:53.437 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:53.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:53.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj93 = obj75 ']' 2026-02-20T18:04:53.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj93 td/test-erasure-eio/CHECK 2026-02-20T18:04:53.456 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:53.457 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:53.457 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj94 = obj75 ']' 2026-02-20T18:04:53.457 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj94 td/test-erasure-eio/CHECK 2026-02-20T18:04:53.475 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:53.476 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:53.476 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj95 = obj75 ']' 2026-02-20T18:04:53.476 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj95 td/test-erasure-eio/CHECK 2026-02-20T18:04:53.494 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:53.495 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:53.495 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj96 = obj75 ']' 2026-02-20T18:04:53.495 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj96 td/test-erasure-eio/CHECK 2026-02-20T18:04:53.513 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:53.514 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:53.514 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj97 = obj75 ']' 2026-02-20T18:04:53.514 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj97 td/test-erasure-eio/CHECK 2026-02-20T18:04:53.533 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:53.534 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:53.534 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj98 = obj75 ']' 2026-02-20T18:04:53.534 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj98 td/test-erasure-eio/CHECK 2026-02-20T18:04:53.552 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:53.553 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:53.553 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj99 = obj75 ']' 2026-02-20T18:04:53.553 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj99 td/test-erasure-eio/CHECK 2026-02-20T18:04:53.571 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:53.572 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-20T18:04:53.572 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj100 = obj75 ']' 2026-02-20T18:04:53.572 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj100 td/test-erasure-eio/CHECK 2026-02-20T18:04:53.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:53.591 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:711: TEST_ec_recovery_unfound: rm -f td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-20T18:04:53.592 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:713: TEST_ec_recovery_unfound: delete_erasure_coded_pool pool-jerasure 2026-02-20T18:04:53.592 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-20T18:04:53.592 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-20T18:04:53.867 INFO:tasks.workunit.client.0.trial195.stderr:pool 'pool-jerasure' does not exist 2026-02-20T18:04:53.875 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-20T18:04:54.153 INFO:tasks.workunit.client.0.trial195.stderr:erasure-code-profile myprofile does not exist 2026-02-20T18:04:54.161 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-20T18:04:54.162 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-20T18:04:54.162 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T18:04:54.162 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-20T18:04:54.162 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:04:54.162 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:04:54.163 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:04:54.163 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:04:54.163 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:04:54.272 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:04:54.273 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T18:04:54.274 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T18:04:54.274 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T18:04:54.275 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T18:04:54.275 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T18:04:54.276 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T18:04:54.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:04:54.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T18:04:54.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T18:04:54.277 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:04:54.278 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T18:04:54.279 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T18:04:54.279 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-20T18:04:54.299 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T18:04:54.299 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:04:54.299 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:04:54.299 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.85808 2026-02-20T18:04:54.302 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T18:04:54.302 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T18:04:54.302 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-20T18:04:54.303 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-20T18:04:54.303 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-20T18:04:54.303 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-20T18:04:54.303 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-20T18:04:54.303 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T18:04:54.303 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-20T18:04:54.303 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:04:54.303 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:04:54.303 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:04:54.303 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:04:54.303 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:04:54.304 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:04:54.304 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T18:04:54.304 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T18:04:54.305 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T18:04:54.305 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T18:04:54.305 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T18:04:54.306 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T18:04:54.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:04:54.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T18:04:54.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T18:04:54.307 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:04:54.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T18:04:54.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T18:04:54.309 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-20T18:04:54.310 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T18:04:54.310 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:04:54.310 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:04:54.310 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.85808 2026-02-20T18:04:54.311 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T18:04:54.311 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T18:04:54.311 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-20T18:04:54.312 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-20T18:04:54.312 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:04:54.312 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:04:54.313 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.85808 2026-02-20T18:04:54.315 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-20T18:04:54.315 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-20T18:04:54.315 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-20T18:04:54.315 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-20T18:04:54.315 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-20T18:04:54.315 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-20T18:04:54.315 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-20T18:04:54.315 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-20T18:04:54.315 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-20T18:04:54.315 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-20T18:04:54.315 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-20T18:04:54.335 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-20T18:04:54.335 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:04:54.335 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:04:54.336 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:04:54.336 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:04:54.336 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:04:54.336 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:04:54.337 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-20T18:04:54.363 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-20T18:04:54.364 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-20T18:04:54.365 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T18:04:54.365 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T18:04:54.365 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-20T18:04:54.365 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-20T18:04:54.366 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T18:04:54.366 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:04:54.366 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:04:54.366 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:04:54.367 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:04:54.367 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:04:54.367 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:04:54.367 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T18:04:54.367 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.85808/ceph-mon.a.asok config get fsid 2026-02-20T18:04:54.409 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-20T18:04:54.409 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T18:04:54.409 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T18:04:54.409 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-20T18:04:54.409 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-20T18:04:54.410 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T18:04:54.410 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:04:54.410 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:04:54.410 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:04:54.410 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:04:54.410 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:04:54.410 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:04:54.410 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T18:04:54.410 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.85808/ceph-mon.a.asok config get mon_host 2026-02-20T18:04:54.456 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-20T18:04:54.456 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-20T18:04:54.457 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-20T18:04:54.457 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-20T18:04:54.457 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-20T18:04:54.457 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-20T18:04:54.457 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-20T18:04:54.561 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-20T18:04:54.561 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:04:54.561 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:04:54.561 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:04:54.561 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:04:54.561 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:04:54.562 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:04:54.562 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T18:04:54.562 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T18:04:54.580 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-20T18:04:54.580 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-20T18:04:54.678 INFO:tasks.workunit.client.0.trial195.stderr:pool 'rbd' already exists 2026-02-20T18:04:54.684 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-20T18:04:55.687 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-20T18:04:55.687 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:04:55.687 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:04:55.687 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:04:55.687 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:04:55.688 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:04:55.688 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:04:55.688 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-20T18:04:55.688 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-mon.a.asok log flush 2026-02-20T18:04:55.729 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T18:04:55.733 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-20T18:04:55.734 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T18:04:54.354+0000 7f0514ab2d80 0 load: jerasure load: lrc 2026-02-20T18:04:55.735 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_single_recovery_error td/test-erasure-eio 2026-02-20T18:04:55.735 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:410: TEST_ec_single_recovery_error: local dir=td/test-erasure-eio 2026-02-20T18:04:55.735 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:411: TEST_ec_single_recovery_error: local objname=myobject 2026-02-20T18:04:55.735 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:413: TEST_ec_single_recovery_error: setup_osds 7 2026-02-20T18:04:55.736 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=7 2026-02-20T18:04:55.736 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-20T18:04:55.736 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 7 - 1 2026-02-20T18:04:55.740 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 6 2026-02-20T18:04:55.740 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:04:55.740 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-20T18:04:55.740 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:04:55.740 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:04:55.740 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-20T18:04:55.740 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:04:55.741 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-20T18:04:55.741 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:04:55.741 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:04:55.741 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:04:55.741 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:04:55.741 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-20T18:04:55.741 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-20T18:04:55.741 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:04:55.741 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:04:55.741 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:04:55.741 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:04:55.741 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:04:55.742 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:04:55.742 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:04:55.742 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:04:55.742 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:04:55.742 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:04:55.742 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:04:55.742 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:04:55.742 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:04:55.742 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:04:55.742 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:04:55.742 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:04:55.742 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:04:55.742 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:04:55.743 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:04:55.743 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:04:55.743 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:04:55.743 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:04:55.743 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-20T18:04:55.743 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:04:55.743 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=00b6cd32-8931-4218-a9f3-3aa9d0c39d34 2026-02-20T18:04:55.743 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 00b6cd32-8931-4218-a9f3-3aa9d0c39d34' 2026-02-20T18:04:55.743 INFO:tasks.workunit.client.0.trial195.stdout:add osd0 00b6cd32-8931-4218-a9f3-3aa9d0c39d34 2026-02-20T18:04:55.744 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:04:55.752 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBHophpWxLBLBAAKfjRlbCvrI6hYgHudYpE2w== 2026-02-20T18:04:55.752 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBHophpWxLBLBAAKfjRlbCvrI6hYgHudYpE2w=="}' 2026-02-20T18:04:55.752 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 00b6cd32-8931-4218-a9f3-3aa9d0c39d34 -i td/test-erasure-eio/0/new.json 2026-02-20T18:04:55.857 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:04:55.864 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-20T18:04:55.865 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBHophpWxLBLBAAKfjRlbCvrI6hYgHudYpE2w== --osd-uuid 00b6cd32-8931-4218-a9f3-3aa9d0c39d34 2026-02-20T18:04:55.884 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:55.882+0000 7f62e4386900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:55.885 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:55.883+0000 7f62e4386900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:55.886 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:55.884+0000 7f62e4386900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:55.886 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:55.884+0000 7f62e4386900 -1 bdev(0x55b4758b8c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-20T18:04:55.887 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:55.884+0000 7f62e4386900 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-20T18:04:56.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-20T18:04:56.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:04:56.323 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-20T18:04:56.323 INFO:tasks.workunit.client.0.trial195.stdout:adding osd0 key to auth repository 2026-02-20T18:04:56.324 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:04:56.430 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-20T18:04:56.431 INFO:tasks.workunit.client.0.trial195.stdout:start osd.0 2026-02-20T18:04:56.432 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:04:56.432 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:04:56.432 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:04:56.432 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:04:56.448 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:56.446+0000 7fa0dbd2d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:56.449 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:56.448+0000 7fa0dbd2d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:56.451 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:56.448+0000 7fa0dbd2d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:56.529 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-20T18:04:56.529 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:04:56.529 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-20T18:04:56.529 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:04:56.529 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:04:56.530 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:04:56.530 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:04:56.531 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:04:56.531 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:04:56.531 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T18:04:56.629 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:04:56.642 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:56.640+0000 7fa0dbd2d900 -1 Falling back to public interface 2026-02-20T18:04:56.896 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:56.894+0000 7fa0dbd2d900 -1 osd.0 0 log_to_monitors true 2026-02-20T18:04:57.630 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:04:57.631 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:04:57.631 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:04:57.632 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:04:57.632 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:04:57.632 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T18:04:57.852 INFO:tasks.workunit.client.0.trial195.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/568239068,v1:127.0.0.1:6803/568239068] [v2:127.0.0.1:6804/568239068,v1:127.0.0.1:6805/568239068] exists,up 00b6cd32-8931-4218-a9f3-3aa9d0c39d34 2026-02-20T18:04:57.856 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:04:57.856 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:04:57.856 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:04:57.856 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:04:57.856 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-20T18:04:57.856 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:04:57.856 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:04:57.856 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-20T18:04:57.856 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:04:57.856 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-20T18:04:57.857 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:04:57.857 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:04:57.857 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:04:57.857 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:04:57.857 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-20T18:04:57.857 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-20T18:04:57.857 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:04:57.857 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:04:57.857 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:04:57.857 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:04:57.857 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:04:57.857 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:04:57.858 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:04:57.858 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:04:57.858 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:04:57.858 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:04:57.858 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:04:57.858 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:04:57.858 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:04:57.858 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:04:57.858 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:04:57.858 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:04:57.859 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:04:57.859 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:04:57.859 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:04:57.859 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:04:57.859 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:04:57.859 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:04:57.860 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-20T18:04:57.860 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:04:57.860 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=168914c2-c8d8-4f76-9867-930a7e8b71b2 2026-02-20T18:04:57.860 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 168914c2-c8d8-4f76-9867-930a7e8b71b2' 2026-02-20T18:04:57.861 INFO:tasks.workunit.client.0.trial195.stdout:add osd1 168914c2-c8d8-4f76-9867-930a7e8b71b2 2026-02-20T18:04:57.861 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:04:57.868 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBJophp1gywMxAAmCleSVwV8CMqN/wkMle9oQ== 2026-02-20T18:04:57.868 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBJophp1gywMxAAmCleSVwV8CMqN/wkMle9oQ=="}' 2026-02-20T18:04:57.869 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 168914c2-c8d8-4f76-9867-930a7e8b71b2 -i td/test-erasure-eio/1/new.json 2026-02-20T18:04:58.093 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:04:58.101 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-20T18:04:58.102 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBJophp1gywMxAAmCleSVwV8CMqN/wkMle9oQ== --osd-uuid 168914c2-c8d8-4f76-9867-930a7e8b71b2 2026-02-20T18:04:58.119 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:58.116+0000 7faacd221900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:58.120 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:58.118+0000 7faacd221900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:58.121 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:58.119+0000 7faacd221900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:58.122 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:58.119+0000 7faacd221900 -1 bdev(0x55bf7d172c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-20T18:04:58.122 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:58.119+0000 7faacd221900 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-20T18:04:58.443 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-20T18:04:58.443 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:04:58.444 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-20T18:04:58.444 INFO:tasks.workunit.client.0.trial195.stdout:adding osd1 key to auth repository 2026-02-20T18:04:58.444 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:04:58.764 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-20T18:04:58.765 INFO:tasks.workunit.client.0.trial195.stdout:start osd.1 2026-02-20T18:04:58.765 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:04:58.765 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:04:58.766 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:04:58.766 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:04:58.781 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:58.778+0000 7f08451da900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:58.782 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:58.780+0000 7f08451da900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:58.783 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:58.781+0000 7f08451da900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:04:58.979 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:58.977+0000 7f08451da900 -1 Falling back to public interface 2026-02-20T18:04:58.991 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-20T18:04:58.991 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:04:58.992 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-20T18:04:58.992 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:04:58.992 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:04:58.992 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:04:58.992 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:04:58.992 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:04:58.993 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:04:58.993 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:04:59.213 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:04:59.338 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:04:59.336+0000 7f08451da900 -1 osd.1 0 log_to_monitors true 2026-02-20T18:05:00.215 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:05:00.215 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:05:00.216 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:05:00.216 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:05:00.216 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:05:00.216 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:05:00.442 INFO:tasks.workunit.client.0.trial195.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/92389594,v1:127.0.0.1:6811/92389594] [v2:127.0.0.1:6812/92389594,v1:127.0.0.1:6813/92389594] exists,up 168914c2-c8d8-4f76-9867-930a7e8b71b2 2026-02-20T18:05:00.446 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:05:00.446 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:05:00.446 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:05:00.446 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:05:00.446 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-20T18:05:00.446 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:05:00.446 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:05:00.447 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-20T18:05:00.447 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:05:00.447 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-20T18:05:00.447 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:05:00.447 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:05:00.447 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:05:00.447 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:05:00.447 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-20T18:05:00.447 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-20T18:05:00.447 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:05:00.448 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:05:00.448 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:05:00.448 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:05:00.448 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:05:00.448 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:05:00.448 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:05:00.448 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:05:00.448 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:05:00.448 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:05:00.448 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:05:00.449 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:05:00.449 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:05:00.449 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:05:00.449 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:05:00.449 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:05:00.449 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:05:00.449 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:05:00.449 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:05:00.449 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:05:00.449 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:05:00.449 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:05:00.449 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-20T18:05:00.449 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:05:00.450 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1305fc1a-9f1b-42c7-8dc0-e5c0ff6c9454 2026-02-20T18:05:00.450 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 1305fc1a-9f1b-42c7-8dc0-e5c0ff6c9454' 2026-02-20T18:05:00.450 INFO:tasks.workunit.client.0.trial195.stdout:add osd2 1305fc1a-9f1b-42c7-8dc0-e5c0ff6c9454 2026-02-20T18:05:00.450 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:05:00.460 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBMophpHF5QGxAA774mjl1IjfZxtx+hAzxOiw== 2026-02-20T18:05:00.460 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBMophpHF5QGxAA774mjl1IjfZxtx+hAzxOiw=="}' 2026-02-20T18:05:00.460 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1305fc1a-9f1b-42c7-8dc0-e5c0ff6c9454 -i td/test-erasure-eio/2/new.json 2026-02-20T18:05:00.690 INFO:tasks.workunit.client.0.trial195.stdout:2 2026-02-20T18:05:00.698 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-20T18:05:00.699 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBMophpHF5QGxAA774mjl1IjfZxtx+hAzxOiw== --osd-uuid 1305fc1a-9f1b-42c7-8dc0-e5c0ff6c9454 2026-02-20T18:05:00.717 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:00.715+0000 7fc862aab900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:00.718 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:00.716+0000 7fc862aab900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:00.719 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:00.717+0000 7fc862aab900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:00.720 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:00.718+0000 7fc862aab900 -1 bdev(0x5628a04eac00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-20T18:05:00.720 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:00.718+0000 7fc862aab900 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-20T18:05:01.049 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-20T18:05:01.049 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:05:01.049 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-20T18:05:01.050 INFO:tasks.workunit.client.0.trial195.stdout:adding osd2 key to auth repository 2026-02-20T18:05:01.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:05:01.366 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-20T18:05:01.366 INFO:tasks.workunit.client.0.trial195.stdout:start osd.2 2026-02-20T18:05:01.366 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:05:01.366 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:05:01.366 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:05:01.366 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:05:01.384 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:01.382+0000 7fb383283900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:01.385 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:01.383+0000 7fb383283900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:01.386 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:01.384+0000 7fb383283900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:01.522 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:01.520+0000 7fb383283900 -1 Falling back to public interface 2026-02-20T18:05:01.593 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-20T18:05:01.594 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:05:01.594 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-20T18:05:01.594 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:05:01.594 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:05:01.594 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:05:01.594 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:05:01.594 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:05:01.595 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:05:01.595 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:05:01.815 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:05:01.836 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:01.834+0000 7fb383283900 -1 osd.2 0 log_to_monitors true 2026-02-20T18:05:02.816 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:05:02.816 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:05:02.817 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:05:02.818 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:05:02.818 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:05:02.818 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:05:03.041 INFO:tasks.workunit.client.0.trial195.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3683424636,v1:127.0.0.1:6819/3683424636] [v2:127.0.0.1:6820/3683424636,v1:127.0.0.1:6821/3683424636] exists,up 1305fc1a-9f1b-42c7-8dc0-e5c0ff6c9454 2026-02-20T18:05:03.046 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:05:03.046 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:05:03.046 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:05:03.046 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:05:03.046 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-20T18:05:03.047 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:05:03.047 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:05:03.047 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-20T18:05:03.047 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:05:03.047 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-20T18:05:03.047 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:05:03.047 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:05:03.047 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:05:03.047 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:05:03.047 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-20T18:05:03.047 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-20T18:05:03.047 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:05:03.048 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:05:03.048 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:05:03.048 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:05:03.048 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:05:03.048 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:05:03.048 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:05:03.048 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:05:03.048 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:05:03.048 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:05:03.048 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:05:03.048 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:05:03.048 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:05:03.049 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:05:03.049 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:05:03.049 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:05:03.049 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:05:03.049 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:05:03.049 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:05:03.049 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:05:03.049 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:05:03.049 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:05:03.049 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-20T18:05:03.049 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:05:03.049 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ad5eb6bd-fd03-43b6-b47e-4be2c7f2ec78 2026-02-20T18:05:03.049 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 ad5eb6bd-fd03-43b6-b47e-4be2c7f2ec78' 2026-02-20T18:05:03.050 INFO:tasks.workunit.client.0.trial195.stdout:add osd3 ad5eb6bd-fd03-43b6-b47e-4be2c7f2ec78 2026-02-20T18:05:03.050 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:05:03.057 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBPophpCJdcAxAAclI5eWwBLki7WUyFopYnzA== 2026-02-20T18:05:03.057 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBPophpCJdcAxAAclI5eWwBLki7WUyFopYnzA=="}' 2026-02-20T18:05:03.057 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ad5eb6bd-fd03-43b6-b47e-4be2c7f2ec78 -i td/test-erasure-eio/3/new.json 2026-02-20T18:05:03.282 INFO:tasks.workunit.client.0.trial195.stdout:3 2026-02-20T18:05:03.289 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-20T18:05:03.290 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBPophpCJdcAxAAclI5eWwBLki7WUyFopYnzA== --osd-uuid ad5eb6bd-fd03-43b6-b47e-4be2c7f2ec78 2026-02-20T18:05:03.309 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:03.307+0000 7fc446eb4900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:03.311 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:03.309+0000 7fc446eb4900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:03.312 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:03.310+0000 7fc446eb4900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:03.312 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:03.310+0000 7fc446eb4900 -1 bdev(0x5650ec7eec00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-20T18:05:03.312 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:03.310+0000 7fc446eb4900 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-20T18:05:03.623 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-20T18:05:03.623 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:05:03.624 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-20T18:05:03.624 INFO:tasks.workunit.client.0.trial195.stdout:adding osd3 key to auth repository 2026-02-20T18:05:03.624 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:05:03.940 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-20T18:05:03.941 INFO:tasks.workunit.client.0.trial195.stdout:start osd.3 2026-02-20T18:05:03.941 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:05:03.942 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:05:03.942 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:05:03.942 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:05:03.957 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:03.955+0000 7fb3e1091900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:03.958 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:03.957+0000 7fb3e1091900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:03.959 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:03.957+0000 7fb3e1091900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:04.085 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:04.083+0000 7fb3e1091900 -1 Falling back to public interface 2026-02-20T18:05:04.165 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-20T18:05:04.165 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:05:04.165 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-20T18:05:04.165 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:05:04.165 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:05:04.165 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:05:04.165 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:05:04.165 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:05:04.165 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:05:04.165 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T18:05:04.384 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:04.382+0000 7fb3e1091900 -1 osd.3 0 log_to_monitors true 2026-02-20T18:05:04.395 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:05:05.397 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:05:05.398 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:05:05.398 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:05:05.398 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:05:05.399 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:05:05.399 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T18:05:05.619 INFO:tasks.workunit.client.0.trial195.stdout:osd.3 up in weight 1 up_from 27 up_thru 28 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2552406433,v1:127.0.0.1:6827/2552406433] [v2:127.0.0.1:6828/2552406433,v1:127.0.0.1:6829/2552406433] exists,up ad5eb6bd-fd03-43b6-b47e-4be2c7f2ec78 2026-02-20T18:05:05.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:05:05.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:05:05.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:05:05.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:05:05.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-20T18:05:05.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:05:05.621 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:05:05.621 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-20T18:05:05.621 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:05:05.621 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-20T18:05:05.621 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:05:05.621 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:05:05.621 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:05:05.621 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:05:05.621 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-20T18:05:05.621 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-20T18:05:05.621 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:05:05.621 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:05:05.621 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:05:05.621 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:05:05.621 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:05:05.621 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:05:05.621 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:05:05.621 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:05:05.621 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:05:05.622 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:05:05.622 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:05:05.622 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:05:05.622 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:05:05.622 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:05:05.622 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:05:05.622 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:05:05.622 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:05:05.622 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:05:05.622 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:05:05.622 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:05:05.622 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:05:05.622 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:05:05.622 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-20T18:05:05.622 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:05:05.623 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d3a9cc36-6254-4886-a23a-5803ca61c421 2026-02-20T18:05:05.623 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 d3a9cc36-6254-4886-a23a-5803ca61c421' 2026-02-20T18:05:05.623 INFO:tasks.workunit.client.0.trial195.stdout:add osd4 d3a9cc36-6254-4886-a23a-5803ca61c421 2026-02-20T18:05:05.623 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:05:05.635 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBRophpxuTPJRAAYFCTdOcNpfb3HLVUygYHAA== 2026-02-20T18:05:05.635 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBRophpxuTPJRAAYFCTdOcNpfb3HLVUygYHAA=="}' 2026-02-20T18:05:05.635 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d3a9cc36-6254-4886-a23a-5803ca61c421 -i td/test-erasure-eio/4/new.json 2026-02-20T18:05:05.862 INFO:tasks.workunit.client.0.trial195.stdout:4 2026-02-20T18:05:05.869 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-20T18:05:05.870 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBRophpxuTPJRAAYFCTdOcNpfb3HLVUygYHAA== --osd-uuid d3a9cc36-6254-4886-a23a-5803ca61c421 2026-02-20T18:05:05.889 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:05.887+0000 7f57cdafe900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:05.890 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:05.888+0000 7f57cdafe900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:05.891 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:05.889+0000 7f57cdafe900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:05.892 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:05.890+0000 7f57cdafe900 -1 bdev(0x5556ec12ec00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-20T18:05:05.892 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:05.890+0000 7f57cdafe900 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-20T18:05:06.220 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-20T18:05:06.221 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:05:06.221 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-20T18:05:06.221 INFO:tasks.workunit.client.0.trial195.stdout:adding osd4 key to auth repository 2026-02-20T18:05:06.222 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:05:06.550 INFO:tasks.workunit.client.0.trial195.stdout:start osd.4 2026-02-20T18:05:06.551 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-20T18:05:06.551 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:05:06.551 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:05:06.552 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:05:06.552 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:05:06.568 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:06.566+0000 7f2176c20900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:06.569 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:06.567+0000 7f2176c20900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:06.570 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:06.568+0000 7f2176c20900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:06.703 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:06.701+0000 7f2176c20900 -1 Falling back to public interface 2026-02-20T18:05:06.777 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-20T18:05:06.777 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:05:06.777 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-20T18:05:06.777 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:05:06.777 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:05:06.777 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:05:06.777 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:05:06.778 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:05:06.778 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:05:06.778 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-20T18:05:06.978 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:06.976+0000 7f2176c20900 -1 osd.4 0 log_to_monitors true 2026-02-20T18:05:07.005 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:05:08.006 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:05:08.006 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:05:08.006 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:05:08.006 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:05:08.007 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:05:08.007 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-20T18:05:08.233 INFO:tasks.workunit.client.0.trial195.stdout:osd.4 up in weight 1 up_from 35 up_thru 36 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/798526580,v1:127.0.0.1:6835/798526580] [v2:127.0.0.1:6836/798526580,v1:127.0.0.1:6837/798526580] exists,up d3a9cc36-6254-4886-a23a-5803ca61c421 2026-02-20T18:05:08.233 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:05:08.233 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:05:08.233 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:05:08.233 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:05:08.233 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-20T18:05:08.233 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:05:08.233 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:05:08.233 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-20T18:05:08.233 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:05:08.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-20T18:05:08.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:05:08.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:05:08.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:05:08.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:05:08.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-20T18:05:08.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-20T18:05:08.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:05:08.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:05:08.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:05:08.235 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:05:08.235 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:05:08.235 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:05:08.235 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:05:08.235 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:05:08.235 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:05:08.235 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:05:08.235 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:05:08.235 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:05:08.235 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:05:08.235 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:05:08.235 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:05:08.235 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:05:08.235 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:05:08.235 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:05:08.235 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:05:08.235 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:05:08.236 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:05:08.236 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:05:08.236 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-20T18:05:08.236 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:05:08.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1d869d2f-fbbb-4e84-a3f4-5c9af65a8133 2026-02-20T18:05:08.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 1d869d2f-fbbb-4e84-a3f4-5c9af65a8133' 2026-02-20T18:05:08.237 INFO:tasks.workunit.client.0.trial195.stdout:add osd5 1d869d2f-fbbb-4e84-a3f4-5c9af65a8133 2026-02-20T18:05:08.238 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:05:08.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBUophpJkLUDhAA4HuRqThcbAxh/BQJWTZ4Ng== 2026-02-20T18:05:08.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBUophpJkLUDhAA4HuRqThcbAxh/BQJWTZ4Ng=="}' 2026-02-20T18:05:08.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1d869d2f-fbbb-4e84-a3f4-5c9af65a8133 -i td/test-erasure-eio/5/new.json 2026-02-20T18:05:08.480 INFO:tasks.workunit.client.0.trial195.stdout:5 2026-02-20T18:05:08.488 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-20T18:05:08.489 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBUophpJkLUDhAA4HuRqThcbAxh/BQJWTZ4Ng== --osd-uuid 1d869d2f-fbbb-4e84-a3f4-5c9af65a8133 2026-02-20T18:05:08.506 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:08.504+0000 7f2574f81900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:08.507 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:08.506+0000 7f2574f81900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:08.508 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:08.507+0000 7f2574f81900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:08.509 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:08.507+0000 7f2574f81900 -1 bdev(0x55a22df68c00 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-20T18:05:08.509 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:08.507+0000 7f2574f81900 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-20T18:05:08.844 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-20T18:05:08.844 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:05:08.845 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-20T18:05:08.845 INFO:tasks.workunit.client.0.trial195.stdout:adding osd5 key to auth repository 2026-02-20T18:05:08.846 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:05:09.174 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-20T18:05:09.175 INFO:tasks.workunit.client.0.trial195.stdout:start osd.5 2026-02-20T18:05:09.176 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:05:09.176 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:05:09.176 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:05:09.176 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:05:09.191 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:09.189+0000 7f7ae120c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:09.192 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:09.190+0000 7f7ae120c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:09.193 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:09.191+0000 7f7ae120c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:09.336 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:09.334+0000 7f7ae120c900 -1 Falling back to public interface 2026-02-20T18:05:09.401 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-20T18:05:09.401 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:05:09.401 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-20T18:05:09.401 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:05:09.402 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:05:09.402 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:05:09.402 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:05:09.403 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:05:09.403 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:05:09.404 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-20T18:05:09.621 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:05:09.679 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:09.678+0000 7f7ae120c900 -1 osd.5 0 log_to_monitors true 2026-02-20T18:05:10.623 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:05:10.623 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:05:10.623 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:05:10.623 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:05:10.624 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:05:10.624 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-20T18:05:10.847 INFO:tasks.workunit.client.0.trial195.stdout:osd.5 up in weight 1 up_from 44 up_thru 45 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/3902125873,v1:127.0.0.1:6843/3902125873] [v2:127.0.0.1:6844/3902125873,v1:127.0.0.1:6845/3902125873] exists,up 1d869d2f-fbbb-4e84-a3f4-5c9af65a8133 2026-02-20T18:05:10.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:05:10.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:05:10.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:05:10.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:05:10.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-20T18:05:10.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:05:10.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:05:10.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-20T18:05:10.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:05:10.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-20T18:05:10.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:05:10.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:05:10.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:05:10.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:05:10.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-20T18:05:10.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-20T18:05:10.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:05:10.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:05:10.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:05:10.848 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:05:10.848 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:05:10.848 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:05:10.849 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:05:10.849 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:05:10.849 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:05:10.849 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:05:10.849 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:05:10.849 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:05:10.849 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:05:10.849 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:05:10.849 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:05:10.849 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:05:10.849 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:05:10.849 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:05:10.849 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:05:10.849 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:05:10.849 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:05:10.849 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:05:10.849 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-20T18:05:10.850 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:05:10.851 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=7576f950-13a3-49e1-a52e-4375ee7d3a92 2026-02-20T18:05:10.851 INFO:tasks.workunit.client.0.trial195.stdout:add osd6 7576f950-13a3-49e1-a52e-4375ee7d3a92 2026-02-20T18:05:10.851 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 7576f950-13a3-49e1-a52e-4375ee7d3a92' 2026-02-20T18:05:10.851 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:05:10.863 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBWophpu6ZlMxAAwhllrtBR43pXzEgb2jts+Q== 2026-02-20T18:05:10.863 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBWophpu6ZlMxAAwhllrtBR43pXzEgb2jts+Q=="}' 2026-02-20T18:05:10.864 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 7576f950-13a3-49e1-a52e-4375ee7d3a92 -i td/test-erasure-eio/6/new.json 2026-02-20T18:05:11.089 INFO:tasks.workunit.client.0.trial195.stdout:6 2026-02-20T18:05:11.097 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-20T18:05:11.098 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBWophpu6ZlMxAAwhllrtBR43pXzEgb2jts+Q== --osd-uuid 7576f950-13a3-49e1-a52e-4375ee7d3a92 2026-02-20T18:05:11.116 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:11.115+0000 7fb77241b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:11.118 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:11.116+0000 7fb77241b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:11.119 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:11.117+0000 7fb77241b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:11.119 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:11.117+0000 7fb77241b900 -1 bdev(0x55b8ef41cc00 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-20T18:05:11.119 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:11.118+0000 7fb77241b900 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-20T18:05:11.459 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-20T18:05:11.460 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:05:11.460 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-20T18:05:11.461 INFO:tasks.workunit.client.0.trial195.stdout:adding osd6 key to auth repository 2026-02-20T18:05:11.461 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:05:11.776 INFO:tasks.workunit.client.0.trial195.stdout:start osd.6 2026-02-20T18:05:11.777 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-20T18:05:11.778 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:05:11.778 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:05:11.778 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:05:11.778 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:05:11.793 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:11.791+0000 7fd254b49900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:11.795 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:11.793+0000 7fd254b49900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:11.796 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:11.794+0000 7fd254b49900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:11.960 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:11.957+0000 7fd254b49900 -1 Falling back to public interface 2026-02-20T18:05:12.003 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-20T18:05:12.003 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:05:12.003 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-20T18:05:12.003 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:05:12.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:05:12.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:05:12.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:05:12.004 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:05:12.005 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:05:12.005 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-20T18:05:12.229 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:05:12.282 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:12.280+0000 7fd254b49900 -1 osd.6 0 log_to_monitors true 2026-02-20T18:05:13.230 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:05:13.231 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:05:13.231 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:05:13.231 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:05:13.232 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:05:13.232 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-20T18:05:13.458 INFO:tasks.workunit.client.0.trial195.stdout:osd.6 up in weight 1 up_from 52 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/1968661700,v1:127.0.0.1:6851/1968661700] [v2:127.0.0.1:6852/1968661700,v1:127.0.0.1:6853/1968661700] exists,up 7576f950-13a3-49e1-a52e-4375ee7d3a92 2026-02-20T18:05:13.460 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:05:13.460 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:05:13.460 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:05:13.460 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-20T18:05:13.460 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-20T18:05:13.460 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-20T18:05:13.461 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:05:13.461 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:05:13.461 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:05:13.461 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.0.asok 2026-02-20T18:05:13.461 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-20T18:05:13.461 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-osd.0.asok log flush 2026-02-20T18:05:13.505 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T18:05:13.508 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-20T18:05:13.511 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T18:04:56.643+0000 7fa0dbd2d900 0 load: jerasure load: lrc 2026-02-20T18:05:13.512 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:415: TEST_ec_single_recovery_error: local poolname=pool-jerasure 2026-02-20T18:05:13.512 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:416: TEST_ec_single_recovery_error: create_erasure_coded_pool pool-jerasure 3 2 2026-02-20T18:05:13.513 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-20T18:05:13.513 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-20T18:05:13.513 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-20T18:05:13.513 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-20T18:05:13.513 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-20T18:05:13.513 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-20T18:05:13.513 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-20T18:05:13.796 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-20T18:05:13.796 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-20T18:05:14.171 INFO:tasks.workunit.client.0.trial195.stderr:pool 'pool-jerasure' already exists 2026-02-20T18:05:14.179 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-20T18:05:15.181 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-20T18:05:15.181 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:05:15.182 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:05:15.182 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:05:15.182 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:05:15.183 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:05:15.183 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:05:15.183 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:05:15.183 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:05:15.183 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:05:15.270 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:05:15.271 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:05:15.271 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:05:15.271 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:05:15.271 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:05:15.271 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:05:15.500 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:05:15.500 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:05:15.501 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:05:15.501 INFO:tasks.workunit.client.0.trial195.stderr:3 2026-02-20T18:05:15.501 INFO:tasks.workunit.client.0.trial195.stderr:4 2026-02-20T18:05:15.501 INFO:tasks.workunit.client.0.trial195.stderr:5 2026-02-20T18:05:15.501 INFO:tasks.workunit.client.0.trial195.stderr:6' 2026-02-20T18:05:15.501 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:05:15.501 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:05:15.501 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:05:15.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803782 2026-02-20T18:05:15.568 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803782 2026-02-20T18:05:15.568 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782' 2026-02-20T18:05:15.568 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:05:15.568 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:05:15.633 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574854 2026-02-20T18:05:15.633 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574854 2026-02-20T18:05:15.633 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-55834574854' 2026-02-20T18:05:15.634 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:05:15.634 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:05:15.698 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411333 2026-02-20T18:05:15.699 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411333 2026-02-20T18:05:15.699 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-77309411333' 2026-02-20T18:05:15.699 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:05:15.699 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:05:15.763 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964116997 2026-02-20T18:05:15.763 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964116997 2026-02-20T18:05:15.764 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-77309411333 3-115964116997' 2026-02-20T18:05:15.764 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:05:15.764 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-20T18:05:15.830 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855364 2026-02-20T18:05:15.830 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855364 2026-02-20T18:05:15.830 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-77309411333 3-115964116997 4-150323855364' 2026-02-20T18:05:15.830 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:05:15.830 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-20T18:05:15.895 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=188978561028 2026-02-20T18:05:15.895 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 188978561028 2026-02-20T18:05:15.896 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-77309411333 3-115964116997 4-150323855364 5-188978561028' 2026-02-20T18:05:15.896 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:05:15.896 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-20T18:05:15.960 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=223338299395 2026-02-20T18:05:15.960 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 223338299395 2026-02-20T18:05:15.960 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-77309411333 3-115964116997 4-150323855364 5-188978561028 6-223338299395' 2026-02-20T18:05:15.960 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:05:15.960 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803782 2026-02-20T18:05:15.961 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:05:15.961 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:05:15.962 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803782 2026-02-20T18:05:15.962 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:05:15.963 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803782 2026-02-20T18:05:15.963 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803782' 2026-02-20T18:05:15.963 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.0 seq 25769803782 2026-02-20T18:05:15.964 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:05:16.188 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803780 -lt 25769803782 2026-02-20T18:05:16.188 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:05:17.189 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:05:17.189 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:05:17.423 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803782 -lt 25769803782 2026-02-20T18:05:17.423 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:05:17.424 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574854 2026-02-20T18:05:17.424 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:05:17.425 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:05:17.425 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-55834574854 2026-02-20T18:05:17.425 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:05:17.426 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574854 2026-02-20T18:05:17.426 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 55834574854' 2026-02-20T18:05:17.427 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.1 seq 55834574854 2026-02-20T18:05:17.427 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:05:17.650 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574854 -lt 55834574854 2026-02-20T18:05:17.651 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:05:17.651 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411333 2026-02-20T18:05:17.651 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:05:17.652 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:05:17.652 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-77309411333 2026-02-20T18:05:17.652 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:05:17.653 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=77309411333 2026-02-20T18:05:17.654 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.2 seq 77309411333 2026-02-20T18:05:17.654 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 77309411333' 2026-02-20T18:05:17.654 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:05:17.883 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411333 -lt 77309411333 2026-02-20T18:05:17.883 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:05:17.883 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964116997 2026-02-20T18:05:17.883 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:05:17.884 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:05:17.885 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-115964116997 2026-02-20T18:05:17.885 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:05:17.886 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=115964116997 2026-02-20T18:05:17.886 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.3 seq 115964116997 2026-02-20T18:05:17.886 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 115964116997' 2026-02-20T18:05:17.886 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:05:18.112 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 115964116997 -lt 115964116997 2026-02-20T18:05:18.112 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:05:18.112 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-150323855364 2026-02-20T18:05:18.113 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:05:18.113 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-20T18:05:18.114 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-150323855364 2026-02-20T18:05:18.114 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:05:18.115 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=150323855364 2026-02-20T18:05:18.115 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 150323855364' 2026-02-20T18:05:18.116 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.4 seq 150323855364 2026-02-20T18:05:18.116 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-20T18:05:18.339 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 150323855364 -lt 150323855364 2026-02-20T18:05:18.339 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:05:18.340 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-188978561028 2026-02-20T18:05:18.340 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:05:18.340 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-20T18:05:18.341 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-188978561028 2026-02-20T18:05:18.341 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:05:18.342 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=188978561028 2026-02-20T18:05:18.342 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 188978561028' 2026-02-20T18:05:18.343 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.5 seq 188978561028 2026-02-20T18:05:18.343 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-20T18:05:18.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 188978561028 -lt 188978561028 2026-02-20T18:05:18.568 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:05:18.568 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-223338299395 2026-02-20T18:05:18.568 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:05:18.569 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-20T18:05:18.569 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-223338299395 2026-02-20T18:05:18.570 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:05:18.571 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=223338299395 2026-02-20T18:05:18.571 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 223338299395' 2026-02-20T18:05:18.572 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.6 seq 223338299395 2026-02-20T18:05:18.572 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-20T18:05:18.797 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 223338299396 -lt 223338299395 2026-02-20T18:05:18.798 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:05:18.799 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:05:18.799 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:05:19.114 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:05:19.114 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:05:19.114 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:05:19.114 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:05:19.114 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:05:19.114 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:05:19.114 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:05:19.114 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:05:19.334 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-20T18:05:19.335 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:05:19.335 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:05:19.335 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:05:19.646 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-20T18:05:19.646 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' -1 2026-02-20T18:05:19.647 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-20T18:05:19.647 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=4 2026-02-20T18:05:19.647 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-20T18:05:19.647 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-20T18:05:19.747 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-20T18:05:19.748 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:05:19.748 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:05:19.748 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:05:19.748 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:05:19.748 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:05:19.748 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:05:19.748 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:05:19.970 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-20T18:05:19.971 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:05:19.971 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:05:19.971 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:05:20.283 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-20T18:05:20.283 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' 4 2026-02-20T18:05:20.283 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-20T18:05:20.283 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-20T18:05:20.284 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-20T18:05:20.284 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-20T18:05:20.284 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-20T18:05:20.284 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-20T18:05:20.284 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-20T18:05:20.594 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-20T18:05:20.594 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-20T18:05:20.594 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 13 )) 2026-02-20T18:05:20.594 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-20T18:05:20.594 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-20T18:05:20.796 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-20T18:05:20.796 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:05:20.796 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:05:20.796 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:05:20.796 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:05:20.797 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:05:20.797 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:05:20.797 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:05:21.017 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-20T18:05:21.018 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:05:21.018 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:05:21.018 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:05:21.329 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-20T18:05:21.329 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' 4 2026-02-20T18:05:21.329 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-20T18:05:21.329 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-20T18:05:21.329 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-20T18:05:21.329 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-20T18:05:21.329 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-20T18:05:21.329 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-20T18:05:21.330 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-20T18:05:21.642 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-20T18:05:21.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-20T18:05:21.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 2 >= 13 )) 2026-02-20T18:05:21.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-20T18:05:21.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.4 2026-02-20T18:05:22.044 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-20T18:05:22.044 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:05:22.044 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:05:22.044 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:05:22.044 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:05:22.044 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:05:22.045 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:05:22.045 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:05:22.267 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-20T18:05:22.267 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:05:22.267 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:05:22.267 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:05:22.578 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-20T18:05:22.578 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' 4 2026-02-20T18:05:22.578 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-20T18:05:22.579 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-20T18:05:22.579 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-20T18:05:22.579 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-20T18:05:22.579 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-20T18:05:22.579 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-20T18:05:22.579 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-20T18:05:22.898 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-20T18:05:22.898 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-20T18:05:22.898 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 3 >= 13 )) 2026-02-20T18:05:22.898 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-20T18:05:22.898 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.8 2026-02-20T18:05:23.698 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-20T18:05:23.699 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:05:23.700 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:05:23.700 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:05:23.700 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:05:23.700 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:05:23.700 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:05:23.700 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:05:23.920 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:05:23.920 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:05:23.921 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:05:23.921 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:05:24.233 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:05:24.233 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:05:24.233 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:05:24.233 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:418: TEST_ec_single_recovery_error: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-20T18:05:24.233 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-20T18:05:24.233 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-20T18:05:24.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-20T18:05:24.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:05:24.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-20T18:05:24.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:05:24.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-20T18:05:24.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:05:24.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-20T18:05:24.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:05:24.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-20T18:05:24.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-20T18:05:24.253 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:419: TEST_ec_single_recovery_error: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 0 2026-02-20T18:05:24.253 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=ec 2026-02-20T18:05:24.253 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-20T18:05:24.254 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=data 2026-02-20T18:05:24.254 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-20T18:05:24.254 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=pool-jerasure 2026-02-20T18:05:24.254 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-20T18:05:24.254 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=myobject 2026-02-20T18:05:24.254 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-20T18:05:24.254 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/test-erasure-eio 2026-02-20T18:05:24.254 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-20T18:05:24.254 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=0 2026-02-20T18:05:24.254 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-20T18:05:24.254 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds pool-jerasure myobject 2026-02-20T18:05:24.254 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:05:24.255 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-20T18:05:24.255 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-20T18:05:24.255 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:05:24.482 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T18:05:24.482 INFO:tasks.workunit.client.0.trial195.stderr:5 2026-02-20T18:05:24.482 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:05:24.482 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T18:05:24.482 INFO:tasks.workunit.client.0.trial195.stderr:6' 2026-02-20T18:05:24.483 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2026-02-20T18:05:24.483 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('3' '5' '1' '0' '6') 2026-02-20T18:05:24.483 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-20T18:05:24.483 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=3 2026-02-20T18:05:24.483 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' ec '!=' ec ']' 2026-02-20T18:05:24.483 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/test-erasure-eio/3/type 2026-02-20T18:05:24.484 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-20T18:05:24.484 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-20T18:05:24.484 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-20T18:05:24.484 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-20T18:05:24.484 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-20T18:05:24.484 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-20T18:05:24.485 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-20T18:05:24.485 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-20T18:05:24.485 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-20T18:05:24.485 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-20T18:05:24.485 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:05:24.485 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:05:24.485 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:05:24.485 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.3.asok 2026-02-20T18:05:24.485 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.85808/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-20T18:05:24.536 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-20T18:05:24.536 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-20T18:05:24.538 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-20T18:05:24.538 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.3 2026-02-20T18:05:24.538 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-20T18:05:24.538 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-20T18:05:24.538 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:05:24.538 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:05:24.538 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:05:24.538 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.3.asok 2026-02-20T18:05:24.539 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-20T18:05:24.539 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-osd.3.asok injectdataerr pool-jerasure myobject 0 2026-02-20T18:05:24.589 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:421: TEST_ec_single_recovery_error: get_osds pool-jerasure myobject 2026-02-20T18:05:24.589 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:05:24.589 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-20T18:05:24.589 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-20T18:05:24.590 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:05:24.816 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T18:05:24.816 INFO:tasks.workunit.client.0.trial195.stderr:5 2026-02-20T18:05:24.816 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:05:24.816 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T18:05:24.817 INFO:tasks.workunit.client.0.trial195.stderr:6' 2026-02-20T18:05:24.817 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2026-02-20T18:05:24.817 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:421: TEST_ec_single_recovery_error: initial_osds=('3' '5' '1' '0' '6') 2026-02-20T18:05:24.817 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:421: TEST_ec_single_recovery_error: local -a initial_osds 2026-02-20T18:05:24.817 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:422: TEST_ec_single_recovery_error: local last_osd=6 2026-02-20T18:05:24.817 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:424: TEST_ec_single_recovery_error: kill_daemons td/test-erasure-eio TERM osd.6 2026-02-20T18:05:24.817 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:05:24.817 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:05:24.817 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:05:24.817 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:05:24.818 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:05:24.922 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:05:24.922 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:425: TEST_ec_single_recovery_error: ceph osd down 6 2026-02-20T18:05:25.148 INFO:tasks.workunit.client.0.trial195.stderr:osd.6 is already down. 2026-02-20T18:05:25.156 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:426: TEST_ec_single_recovery_error: ceph osd out 6 2026-02-20T18:05:25.432 INFO:tasks.workunit.client.0.trial195.stderr:osd.6 is already out. 2026-02-20T18:05:25.440 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:429: TEST_ec_single_recovery_error: wait_for_clean 2026-02-20T18:05:25.440 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:05:25.440 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:05:25.440 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:05:25.441 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:05:25.441 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:05:25.441 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:05:25.441 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:05:25.441 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:05:25.441 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:05:25.527 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:05:25.528 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:05:25.528 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:05:25.528 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:05:25.528 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:05:25.528 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:05:25.752 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:05:25.752 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:05:25.752 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:05:25.752 INFO:tasks.workunit.client.0.trial195.stderr:3 2026-02-20T18:05:25.752 INFO:tasks.workunit.client.0.trial195.stderr:4 2026-02-20T18:05:25.752 INFO:tasks.workunit.client.0.trial195.stderr:5 2026-02-20T18:05:25.752 INFO:tasks.workunit.client.0.trial195.stderr:6' 2026-02-20T18:05:25.752 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:05:25.753 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:05:25.753 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:05:25.817 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803786 2026-02-20T18:05:25.817 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803786 2026-02-20T18:05:25.817 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786' 2026-02-20T18:05:25.817 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:05:25.817 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:05:25.881 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574858 2026-02-20T18:05:25.882 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574858 2026-02-20T18:05:25.882 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786 1-55834574858' 2026-02-20T18:05:25.882 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:05:25.882 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:05:25.947 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411337 2026-02-20T18:05:25.947 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411337 2026-02-20T18:05:25.947 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786 1-55834574858 2-77309411337' 2026-02-20T18:05:25.947 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:05:25.947 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:05:26.013 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964117001 2026-02-20T18:05:26.013 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964117001 2026-02-20T18:05:26.013 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786 1-55834574858 2-77309411337 3-115964117001' 2026-02-20T18:05:26.013 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:05:26.013 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-20T18:05:26.077 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855368 2026-02-20T18:05:26.077 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855368 2026-02-20T18:05:26.077 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786 1-55834574858 2-77309411337 3-115964117001 4-150323855368' 2026-02-20T18:05:26.077 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:05:26.078 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-20T18:05:26.143 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=188978561032 2026-02-20T18:05:26.143 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 188978561032 2026-02-20T18:05:26.143 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786 1-55834574858 2-77309411337 3-115964117001 4-150323855368 5-188978561032' 2026-02-20T18:05:26.143 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:05:26.143 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-20T18:05:26.190 INFO:tasks.workunit.client.0.trial195.stderr:Error ENXIO: problem getting command descriptions from osd.6 2026-02-20T18:05:26.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq= 2026-02-20T18:05:26.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z '' 2026-02-20T18:05:26.192 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2261: flush_pg_stats: continue 2026-02-20T18:05:26.192 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:05:26.192 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803786 2026-02-20T18:05:26.192 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:05:26.192 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:05:26.193 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803786 2026-02-20T18:05:26.193 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:05:26.194 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803786 2026-02-20T18:05:26.194 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803786' 2026-02-20T18:05:26.194 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.0 seq 25769803786 2026-02-20T18:05:26.194 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:05:26.421 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803784 -lt 25769803786 2026-02-20T18:05:26.421 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:05:27.421 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:05:27.422 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:05:27.650 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803786 -lt 25769803786 2026-02-20T18:05:27.650 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:05:27.650 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574858 2026-02-20T18:05:27.651 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:05:27.651 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:05:27.652 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-55834574858 2026-02-20T18:05:27.652 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:05:27.653 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574858 2026-02-20T18:05:27.653 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 55834574858' 2026-02-20T18:05:27.654 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.1 seq 55834574858 2026-02-20T18:05:27.654 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:05:27.877 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574858 -lt 55834574858 2026-02-20T18:05:27.877 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:05:27.877 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411337 2026-02-20T18:05:27.878 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:05:27.878 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:05:27.879 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-77309411337 2026-02-20T18:05:27.879 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:05:27.880 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=77309411337 2026-02-20T18:05:27.880 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 77309411337' 2026-02-20T18:05:27.880 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.2 seq 77309411337 2026-02-20T18:05:27.881 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:05:28.106 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411337 -lt 77309411337 2026-02-20T18:05:28.107 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:05:28.107 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964117001 2026-02-20T18:05:28.107 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:05:28.108 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:05:28.109 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-115964117001 2026-02-20T18:05:28.109 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:05:28.110 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=115964117001 2026-02-20T18:05:28.110 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 115964117001' 2026-02-20T18:05:28.110 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.3 seq 115964117001 2026-02-20T18:05:28.110 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:05:28.341 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 115964117001 -lt 115964117001 2026-02-20T18:05:28.341 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:05:28.342 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-150323855368 2026-02-20T18:05:28.342 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:05:28.342 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-20T18:05:28.343 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-150323855368 2026-02-20T18:05:28.343 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:05:28.344 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=150323855368 2026-02-20T18:05:28.344 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 150323855368' 2026-02-20T18:05:28.345 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.4 seq 150323855368 2026-02-20T18:05:28.345 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-20T18:05:28.570 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 150323855368 -lt 150323855368 2026-02-20T18:05:28.571 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:05:28.571 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-188978561032 2026-02-20T18:05:28.571 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:05:28.572 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-20T18:05:28.573 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-188978561032 2026-02-20T18:05:28.573 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:05:28.574 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=188978561032 2026-02-20T18:05:28.574 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 188978561032' 2026-02-20T18:05:28.574 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.5 seq 188978561032 2026-02-20T18:05:28.575 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-20T18:05:28.797 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 188978561032 -lt 188978561032 2026-02-20T18:05:28.797 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:05:28.797 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:05:28.798 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:05:29.109 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:05:29.109 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:05:29.109 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:05:29.109 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:05:29.110 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:05:29.110 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:05:29.110 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:05:29.110 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:05:29.331 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-20T18:05:29.332 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:05:29.332 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:05:29.332 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:05:29.645 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-20T18:05:29.646 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' -1 2026-02-20T18:05:29.646 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-20T18:05:29.646 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=4 2026-02-20T18:05:29.646 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-20T18:05:29.646 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-20T18:05:29.746 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-20T18:05:29.747 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:05:29.748 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:05:29.748 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:05:29.748 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:05:29.748 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:05:29.748 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:05:29.748 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:05:29.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-20T18:05:29.974 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:05:29.974 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:05:29.975 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:05:30.286 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-20T18:05:30.287 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' 4 2026-02-20T18:05:30.287 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-20T18:05:30.287 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-20T18:05:30.287 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-20T18:05:30.287 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-20T18:05:30.287 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-20T18:05:30.287 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-20T18:05:30.287 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-20T18:05:30.600 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-20T18:05:30.600 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-20T18:05:30.600 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 13 )) 2026-02-20T18:05:30.600 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-20T18:05:30.600 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-20T18:05:30.800 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-20T18:05:30.801 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:05:30.801 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:05:30.801 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:05:30.801 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:05:30.801 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:05:30.802 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:05:30.802 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:05:31.025 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-20T18:05:31.026 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:05:31.026 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:05:31.026 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:05:31.336 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-20T18:05:31.336 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' 4 2026-02-20T18:05:31.337 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-20T18:05:31.337 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-20T18:05:31.337 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-20T18:05:31.337 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-20T18:05:31.337 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-20T18:05:31.337 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-20T18:05:31.337 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-20T18:05:31.647 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-20T18:05:31.647 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-20T18:05:31.648 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 2 >= 13 )) 2026-02-20T18:05:31.648 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-20T18:05:31.648 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.4 2026-02-20T18:05:32.049 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-20T18:05:32.049 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:05:32.050 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:05:32.050 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:05:32.050 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:05:32.050 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:05:32.050 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:05:32.050 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:05:32.269 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-20T18:05:32.270 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:05:32.270 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:05:32.270 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:05:32.585 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-20T18:05:32.585 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' 4 2026-02-20T18:05:32.585 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-20T18:05:32.586 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-20T18:05:32.586 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-20T18:05:32.586 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-20T18:05:32.586 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-20T18:05:32.586 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-20T18:05:32.586 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-20T18:05:32.897 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=624 2026-02-20T18:05:32.897 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test 624 '!=' null 2026-02-20T18:05:32.897 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1668: wait_for_clean: loop=0 2026-02-20T18:05:32.897 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-20T18:05:32.897 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-20T18:05:32.999 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-20T18:05:32.999 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:05:33.000 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:05:33.000 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:05:33.000 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:05:33.000 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:05:33.000 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:05:33.001 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:05:33.222 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:05:33.223 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:05:33.223 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:05:33.223 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:05:33.534 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:05:33.534 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:05:33.535 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:05:33.535 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:431: TEST_ec_single_recovery_error: rados_get td/test-erasure-eio pool-jerasure myobject 2026-02-20T18:05:33.535 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-20T18:05:33.535 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-20T18:05:33.535 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=myobject 2026-02-20T18:05:33.535 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-20T18:05:33.535 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-20T18:05:33.535 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get myobject td/test-erasure-eio/COPY 2026-02-20T18:05:33.553 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-20T18:05:33.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-20T18:05:33.555 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:433: TEST_ec_single_recovery_error: delete_erasure_coded_pool pool-jerasure 2026-02-20T18:05:33.555 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-20T18:05:33.555 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-20T18:05:33.836 INFO:tasks.workunit.client.0.trial195.stderr:pool 'pool-jerasure' does not exist 2026-02-20T18:05:33.844 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-20T18:05:34.121 INFO:tasks.workunit.client.0.trial195.stderr:erasure-code-profile myprofile does not exist 2026-02-20T18:05:34.130 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-20T18:05:34.130 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-20T18:05:34.130 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T18:05:34.130 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-20T18:05:34.130 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:05:34.131 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:05:34.131 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:05:34.131 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:05:34.131 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:05:34.240 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:05:34.241 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T18:05:34.242 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T18:05:34.242 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T18:05:34.243 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T18:05:34.243 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T18:05:34.244 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T18:05:34.244 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:05:34.245 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T18:05:34.245 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T18:05:34.245 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:05:34.246 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T18:05:34.247 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T18:05:34.247 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-20T18:05:34.261 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T18:05:34.262 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:05:34.262 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:05:34.262 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.85808 2026-02-20T18:05:34.263 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T18:05:34.263 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T18:05:34.264 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-20T18:05:34.264 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-20T18:05:34.264 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-20T18:05:34.264 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-20T18:05:34.264 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-20T18:05:34.264 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T18:05:34.264 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-20T18:05:34.264 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:05:34.264 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:05:34.264 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:05:34.264 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:05:34.264 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:05:34.265 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:05:34.265 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T18:05:34.266 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T18:05:34.266 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T18:05:34.267 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T18:05:34.267 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T18:05:34.268 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T18:05:34.268 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:05:34.268 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T18:05:34.269 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T18:05:34.269 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:05:34.270 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T18:05:34.270 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T18:05:34.271 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-20T18:05:34.272 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T18:05:34.272 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:05:34.272 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:05:34.272 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.85808 2026-02-20T18:05:34.273 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T18:05:34.273 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T18:05:34.273 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-20T18:05:34.274 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-20T18:05:34.274 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:05:34.274 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:05:34.274 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.85808 2026-02-20T18:05:34.275 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-20T18:05:34.276 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-20T18:05:34.276 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-20T18:05:34.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-20T18:05:34.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-20T18:05:34.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-20T18:05:34.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-20T18:05:34.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-20T18:05:34.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-20T18:05:34.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-20T18:05:34.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-20T18:05:34.296 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-20T18:05:34.296 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:05:34.296 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:05:34.297 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:05:34.297 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:05:34.297 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:05:34.297 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:05:34.297 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-20T18:05:34.324 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-20T18:05:34.325 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-20T18:05:34.325 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T18:05:34.325 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T18:05:34.325 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-20T18:05:34.325 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-20T18:05:34.326 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T18:05:34.326 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:05:34.326 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:05:34.326 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:05:34.326 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:05:34.326 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:05:34.326 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:05:34.326 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T18:05:34.326 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.85808/ceph-mon.a.asok config get fsid 2026-02-20T18:05:34.373 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-20T18:05:34.373 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T18:05:34.373 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T18:05:34.373 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-20T18:05:34.373 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-20T18:05:34.373 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T18:05:34.374 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:05:34.374 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:05:34.374 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:05:34.374 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:05:34.374 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:05:34.374 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:05:34.374 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T18:05:34.374 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.85808/ceph-mon.a.asok config get mon_host 2026-02-20T18:05:34.419 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-20T18:05:34.420 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-20T18:05:34.420 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-20T18:05:34.420 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-20T18:05:34.420 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-20T18:05:34.420 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-20T18:05:34.420 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-20T18:05:34.520 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-20T18:05:34.520 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:05:34.520 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:05:34.521 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:05:34.521 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:05:34.521 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:05:34.521 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:05:34.522 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T18:05:34.522 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T18:05:34.539 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-20T18:05:34.539 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-20T18:05:34.636 INFO:tasks.workunit.client.0.trial195.stderr:pool 'rbd' already exists 2026-02-20T18:05:34.642 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-20T18:05:35.644 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-20T18:05:35.644 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:05:35.644 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:05:35.644 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:05:35.644 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:05:35.644 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:05:35.644 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:05:35.644 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-20T18:05:35.644 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-mon.a.asok log flush 2026-02-20T18:05:35.686 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T18:05:35.691 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-20T18:05:35.692 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T18:05:34.316+0000 7fd0dae0ed80 0 load: jerasure load: lrc 2026-02-20T18:05:35.693 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_bad_size_shard_0 td/test-erasure-eio 2026-02-20T18:05:35.693 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:322: TEST_rados_get_bad_size_shard_0: local dir=td/test-erasure-eio 2026-02-20T18:05:35.693 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:323: TEST_rados_get_bad_size_shard_0: setup_osds 4 2026-02-20T18:05:35.693 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-20T18:05:35.693 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-20T18:05:35.693 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-20T18:05:35.694 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-20T18:05:35.695 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:05:35.695 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-20T18:05:35.695 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:05:35.695 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:05:35.695 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-20T18:05:35.695 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:05:35.695 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-20T18:05:35.695 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:05:35.695 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:05:35.695 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:05:35.695 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:05:35.695 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-20T18:05:35.696 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-20T18:05:35.696 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:05:35.696 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:05:35.696 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:05:35.696 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:05:35.696 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:05:35.696 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:05:35.696 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:05:35.696 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:05:35.696 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:05:35.696 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:05:35.697 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:05:35.697 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:05:35.697 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:05:35.697 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:05:35.697 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:05:35.697 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:05:35.697 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:05:35.697 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:05:35.697 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:05:35.697 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:05:35.697 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:05:35.697 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:05:35.697 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-20T18:05:35.697 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:05:35.697 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1fc99c53-8e78-43df-b2cc-b3a2266e3a82 2026-02-20T18:05:35.697 INFO:tasks.workunit.client.0.trial195.stdout:add osd0 1fc99c53-8e78-43df-b2cc-b3a2266e3a82 2026-02-20T18:05:35.697 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 1fc99c53-8e78-43df-b2cc-b3a2266e3a82' 2026-02-20T18:05:35.698 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:05:35.709 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBvophpHsc+KhAAfHi+GxqTv+jhSGqPt6A08A== 2026-02-20T18:05:35.709 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBvophpHsc+KhAAfHi+GxqTv+jhSGqPt6A08A=="}' 2026-02-20T18:05:35.709 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1fc99c53-8e78-43df-b2cc-b3a2266e3a82 -i td/test-erasure-eio/0/new.json 2026-02-20T18:05:35.815 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:05:35.822 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-20T18:05:35.823 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBvophpHsc+KhAAfHi+GxqTv+jhSGqPt6A08A== --osd-uuid 1fc99c53-8e78-43df-b2cc-b3a2266e3a82 2026-02-20T18:05:35.841 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:35.840+0000 7f2085b60900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:35.843 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:35.843+0000 7f2085b60900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:35.844 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:35.844+0000 7f2085b60900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:35.845 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:35.844+0000 7f2085b60900 -1 bdev(0x55a9dd210c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-20T18:05:35.845 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:35.844+0000 7f2085b60900 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-20T18:05:36.282 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-20T18:05:36.283 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:05:36.283 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-20T18:05:36.283 INFO:tasks.workunit.client.0.trial195.stdout:adding osd0 key to auth repository 2026-02-20T18:05:36.284 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:05:36.387 INFO:tasks.workunit.client.0.trial195.stdout:start osd.0 2026-02-20T18:05:36.387 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-20T18:05:36.387 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:05:36.387 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:05:36.387 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:05:36.387 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:05:36.404 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:36.403+0000 7f7389795900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:36.405 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:36.404+0000 7f7389795900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:36.406 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:36.405+0000 7f7389795900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:36.488 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-20T18:05:36.489 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:05:36.489 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-20T18:05:36.489 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:05:36.489 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:05:36.489 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:05:36.489 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:05:36.490 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:05:36.490 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:05:36.490 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T18:05:36.584 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:05:36.604 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:36.604+0000 7f7389795900 -1 Falling back to public interface 2026-02-20T18:05:36.855 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:36.854+0000 7f7389795900 -1 osd.0 0 log_to_monitors true 2026-02-20T18:05:37.585 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:05:37.586 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:05:37.586 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:05:37.586 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:05:37.586 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:05:37.586 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T18:05:37.811 INFO:tasks.workunit.client.0.trial195.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2981935056,v1:127.0.0.1:6803/2981935056] [v2:127.0.0.1:6804/2981935056,v1:127.0.0.1:6805/2981935056] exists,up 1fc99c53-8e78-43df-b2cc-b3a2266e3a82 2026-02-20T18:05:37.815 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:05:37.816 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:05:37.816 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:05:37.816 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:05:37.816 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-20T18:05:37.816 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:05:37.816 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:05:37.816 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-20T18:05:37.816 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:05:37.816 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-20T18:05:37.816 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:05:37.816 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:05:37.817 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:05:37.817 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:05:37.817 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-20T18:05:37.817 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-20T18:05:37.817 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:05:37.817 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:05:37.817 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:05:37.817 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:05:37.817 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:05:37.817 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:05:37.817 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:05:37.817 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:05:37.818 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:05:37.818 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:05:37.818 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:05:37.818 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:05:37.818 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:05:37.818 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:05:37.818 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:05:37.818 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:05:37.818 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:05:37.818 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:05:37.818 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:05:37.818 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:05:37.819 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:05:37.819 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:05:37.819 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-20T18:05:37.819 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:05:37.819 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=cf55ea6b-c46c-423c-b53a-234bec6c36cc 2026-02-20T18:05:37.819 INFO:tasks.workunit.client.0.trial195.stdout:add osd1 cf55ea6b-c46c-423c-b53a-234bec6c36cc 2026-02-20T18:05:37.820 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 cf55ea6b-c46c-423c-b53a-234bec6c36cc' 2026-02-20T18:05:37.820 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:05:37.828 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBxophpU7lIMRAAxrfwJgR+At0lRF1+yokIHg== 2026-02-20T18:05:37.828 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBxophpU7lIMRAAxrfwJgR+At0lRF1+yokIHg=="}' 2026-02-20T18:05:37.828 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new cf55ea6b-c46c-423c-b53a-234bec6c36cc -i td/test-erasure-eio/1/new.json 2026-02-20T18:05:38.056 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:05:38.064 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-20T18:05:38.065 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBxophpU7lIMRAAxrfwJgR+At0lRF1+yokIHg== --osd-uuid cf55ea6b-c46c-423c-b53a-234bec6c36cc 2026-02-20T18:05:38.083 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:38.082+0000 7fdc2bd7d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:38.085 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:38.084+0000 7fdc2bd7d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:38.086 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:38.085+0000 7fdc2bd7d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:38.086 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:38.085+0000 7fdc2bd7d900 -1 bdev(0x562d7f544c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-20T18:05:38.087 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:38.085+0000 7fdc2bd7d900 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-20T18:05:38.417 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-20T18:05:38.417 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:05:38.418 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-20T18:05:38.418 INFO:tasks.workunit.client.0.trial195.stdout:adding osd1 key to auth repository 2026-02-20T18:05:38.418 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:05:38.737 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-20T18:05:38.738 INFO:tasks.workunit.client.0.trial195.stdout:start osd.1 2026-02-20T18:05:38.739 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:05:38.740 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:05:38.740 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:05:38.740 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:05:38.754 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:38.753+0000 7fb466364900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:38.755 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:38.754+0000 7fb466364900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:38.756 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:38.755+0000 7fb466364900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:38.897 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:38.896+0000 7fb466364900 -1 Falling back to public interface 2026-02-20T18:05:38.964 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-20T18:05:38.964 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:05:38.965 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-20T18:05:38.965 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:05:38.965 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:05:38.965 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:05:38.965 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:05:38.965 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:05:38.966 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:05:38.966 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:05:39.186 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:05:39.258 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:39.257+0000 7fb466364900 -1 osd.1 0 log_to_monitors true 2026-02-20T18:05:40.188 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:05:40.188 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:05:40.188 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:05:40.189 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:05:40.189 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:05:40.189 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:05:40.412 INFO:tasks.workunit.client.0.trial195.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3339457191,v1:127.0.0.1:6811/3339457191] [v2:127.0.0.1:6812/3339457191,v1:127.0.0.1:6813/3339457191] exists,up cf55ea6b-c46c-423c-b53a-234bec6c36cc 2026-02-20T18:05:40.417 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:05:40.417 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:05:40.417 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:05:40.417 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:05:40.417 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-20T18:05:40.417 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:05:40.417 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:05:40.417 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-20T18:05:40.418 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:05:40.418 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-20T18:05:40.418 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:05:40.418 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:05:40.418 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:05:40.418 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:05:40.418 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-20T18:05:40.418 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-20T18:05:40.418 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:05:40.418 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:05:40.418 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:05:40.418 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:05:40.418 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:05:40.418 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:05:40.418 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:05:40.419 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:05:40.419 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:05:40.419 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:05:40.419 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:05:40.419 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:05:40.419 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:05:40.419 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:05:40.419 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:05:40.419 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:05:40.419 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:05:40.419 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:05:40.419 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:05:40.419 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:05:40.419 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:05:40.420 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:05:40.420 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-20T18:05:40.420 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:05:40.420 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=9b751d1a-fad6-4676-a537-335de829397e 2026-02-20T18:05:40.420 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 9b751d1a-fad6-4676-a537-335de829397e' 2026-02-20T18:05:40.420 INFO:tasks.workunit.client.0.trial195.stdout:add osd2 9b751d1a-fad6-4676-a537-335de829397e 2026-02-20T18:05:40.420 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:05:40.429 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB0ophpFd9/GRAAzFZnj8mcKd6AFb+XvukAAA== 2026-02-20T18:05:40.429 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB0ophpFd9/GRAAzFZnj8mcKd6AFb+XvukAAA=="}' 2026-02-20T18:05:40.429 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 9b751d1a-fad6-4676-a537-335de829397e -i td/test-erasure-eio/2/new.json 2026-02-20T18:05:40.655 INFO:tasks.workunit.client.0.trial195.stdout:2 2026-02-20T18:05:40.663 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-20T18:05:40.663 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB0ophpFd9/GRAAzFZnj8mcKd6AFb+XvukAAA== --osd-uuid 9b751d1a-fad6-4676-a537-335de829397e 2026-02-20T18:05:40.681 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:40.680+0000 7fd38d872900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:40.683 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:40.682+0000 7fd38d872900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:40.684 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:40.683+0000 7fd38d872900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:40.684 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:40.683+0000 7fd38d872900 -1 bdev(0x55ea0790ac00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-20T18:05:40.684 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:40.683+0000 7fd38d872900 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-20T18:05:40.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-20T18:05:40.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:05:40.997 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-20T18:05:40.997 INFO:tasks.workunit.client.0.trial195.stdout:adding osd2 key to auth repository 2026-02-20T18:05:40.997 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:05:41.313 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-20T18:05:41.313 INFO:tasks.workunit.client.0.trial195.stdout:start osd.2 2026-02-20T18:05:41.314 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:05:41.314 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:05:41.314 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:05:41.314 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:05:41.330 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:41.330+0000 7f98a7022900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:41.332 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:41.331+0000 7f98a7022900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:41.333 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:41.332+0000 7f98a7022900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:41.474 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:41.473+0000 7f98a7022900 -1 Falling back to public interface 2026-02-20T18:05:41.540 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-20T18:05:41.540 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:05:41.540 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-20T18:05:41.540 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:05:41.540 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:05:41.540 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:05:41.540 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:05:41.540 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:05:41.541 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:05:41.541 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:05:41.765 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:05:41.769 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:41.768+0000 7f98a7022900 -1 osd.2 0 log_to_monitors true 2026-02-20T18:05:42.766 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:05:42.766 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:05:42.766 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:05:42.767 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:05:42.767 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:05:42.767 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:05:42.997 INFO:tasks.workunit.client.0.trial195.stdout:osd.2 up in weight 1 up_from 17 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1581565668,v1:127.0.0.1:6819/1581565668] [v2:127.0.0.1:6820/1581565668,v1:127.0.0.1:6821/1581565668] exists,up 9b751d1a-fad6-4676-a537-335de829397e 2026-02-20T18:05:43.001 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:05:43.001 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:05:43.001 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:05:43.001 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:05:43.001 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-20T18:05:43.001 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:05:43.001 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:05:43.002 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-20T18:05:43.002 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:05:43.002 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-20T18:05:43.002 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:05:43.002 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:05:43.002 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:05:43.002 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:05:43.002 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-20T18:05:43.002 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-20T18:05:43.002 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:05:43.002 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:05:43.002 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:05:43.003 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:05:43.003 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:05:43.003 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:05:43.003 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:05:43.003 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:05:43.003 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:05:43.003 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:05:43.003 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:05:43.003 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:05:43.003 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:05:43.003 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:05:43.003 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:05:43.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:05:43.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:05:43.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:05:43.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:05:43.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:05:43.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:05:43.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:05:43.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-20T18:05:43.004 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:05:43.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=90b62f9b-4234-48a4-9308-02e49957e131 2026-02-20T18:05:43.005 INFO:tasks.workunit.client.0.trial195.stdout:add osd3 90b62f9b-4234-48a4-9308-02e49957e131 2026-02-20T18:05:43.005 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 90b62f9b-4234-48a4-9308-02e49957e131' 2026-02-20T18:05:43.005 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:05:43.013 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB3ophpY+G1ABAA6gslKizJW1gN8EX+JhdlFw== 2026-02-20T18:05:43.013 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB3ophpY+G1ABAA6gslKizJW1gN8EX+JhdlFw=="}' 2026-02-20T18:05:43.013 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 90b62f9b-4234-48a4-9308-02e49957e131 -i td/test-erasure-eio/3/new.json 2026-02-20T18:05:43.240 INFO:tasks.workunit.client.0.trial195.stdout:3 2026-02-20T18:05:43.248 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-20T18:05:43.249 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB3ophpY+G1ABAA6gslKizJW1gN8EX+JhdlFw== --osd-uuid 90b62f9b-4234-48a4-9308-02e49957e131 2026-02-20T18:05:43.267 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:43.266+0000 7fd1dcc23900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:43.269 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:43.268+0000 7fd1dcc23900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:43.269 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:43.269+0000 7fd1dcc23900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:43.270 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:43.269+0000 7fd1dcc23900 -1 bdev(0x55dfede30c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-20T18:05:43.270 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:43.269+0000 7fd1dcc23900 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-20T18:05:43.603 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-20T18:05:43.603 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:05:43.603 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-20T18:05:43.604 INFO:tasks.workunit.client.0.trial195.stdout:adding osd3 key to auth repository 2026-02-20T18:05:43.604 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:05:43.920 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-20T18:05:43.920 INFO:tasks.workunit.client.0.trial195.stdout:start osd.3 2026-02-20T18:05:43.920 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:05:43.920 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:05:43.920 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:05:43.920 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:05:43.937 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:43.936+0000 7fd7a4623900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:43.939 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:43.938+0000 7fd7a4623900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:43.940 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:43.939+0000 7fd7a4623900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:44.068 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:44.067+0000 7fd7a4623900 -1 Falling back to public interface 2026-02-20T18:05:44.144 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-20T18:05:44.144 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:05:44.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-20T18:05:44.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:05:44.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:05:44.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:05:44.145 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:05:44.146 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:05:44.146 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:05:44.146 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T18:05:44.374 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:05:44.388 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:44.387+0000 7fd7a4623900 -1 osd.3 0 log_to_monitors true 2026-02-20T18:05:45.376 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:05:45.376 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:05:45.376 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:05:45.376 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:05:45.377 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:05:45.377 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T18:05:45.599 INFO:tasks.workunit.client.0.trial195.stdout:osd.3 up in weight 1 up_from 25 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1939790094,v1:127.0.0.1:6827/1939790094] [v2:127.0.0.1:6828/1939790094,v1:127.0.0.1:6829/1939790094] exists,up 90b62f9b-4234-48a4-9308-02e49957e131 2026-02-20T18:05:45.601 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:05:45.601 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:05:45.601 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:05:45.601 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-20T18:05:45.601 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-20T18:05:45.601 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-20T18:05:45.601 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:05:45.601 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:05:45.601 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:05:45.602 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.0.asok 2026-02-20T18:05:45.602 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-20T18:05:45.602 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-osd.0.asok log flush 2026-02-20T18:05:45.647 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T18:05:45.651 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-20T18:05:45.653 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T18:05:36.607+0000 7f7389795900 0 load: jerasure load: lrc 2026-02-20T18:05:45.654 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:325: TEST_rados_get_bad_size_shard_0: local poolname=pool-jerasure 2026-02-20T18:05:45.655 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:326: TEST_rados_get_bad_size_shard_0: create_erasure_coded_pool pool-jerasure 2 1 2026-02-20T18:05:45.655 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-20T18:05:45.655 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-20T18:05:45.655 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-20T18:05:45.655 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-20T18:05:45.655 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-20T18:05:45.655 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-20T18:05:45.655 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-20T18:05:45.940 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-20T18:05:45.940 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-20T18:05:46.313 INFO:tasks.workunit.client.0.trial195.stderr:pool 'pool-jerasure' already exists 2026-02-20T18:05:46.321 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-20T18:05:47.323 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-20T18:05:47.323 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:05:47.323 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:05:47.323 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:05:47.323 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:05:47.324 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:05:47.324 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:05:47.324 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:05:47.324 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:05:47.324 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:05:47.411 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:05:47.411 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:05:47.411 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:05:47.411 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:05:47.411 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:05:47.411 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:05:47.636 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:05:47.636 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:05:47.636 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:05:47.636 INFO:tasks.workunit.client.0.trial195.stderr:3' 2026-02-20T18:05:47.636 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:05:47.636 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:05:47.636 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:05:47.700 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803780 2026-02-20T18:05:47.700 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803780 2026-02-20T18:05:47.701 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803780' 2026-02-20T18:05:47.701 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:05:47.701 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:05:47.768 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607556 2026-02-20T18:05:47.768 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607556 2026-02-20T18:05:47.768 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803780 1-51539607556' 2026-02-20T18:05:47.768 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:05:47.769 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:05:47.833 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444035 2026-02-20T18:05:47.833 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444035 2026-02-20T18:05:47.833 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803780 1-51539607556 2-73014444035' 2026-02-20T18:05:47.833 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:05:47.833 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:05:47.896 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182404 2026-02-20T18:05:47.896 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182404 2026-02-20T18:05:47.897 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803780 1-51539607556 2-73014444035 3-107374182404' 2026-02-20T18:05:47.897 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:05:47.897 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803780 2026-02-20T18:05:47.897 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:05:47.897 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:05:47.898 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803780 2026-02-20T18:05:47.898 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:05:47.899 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803780 2026-02-20T18:05:47.900 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.0 seq 25769803780 2026-02-20T18:05:47.900 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803780' 2026-02-20T18:05:47.900 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:05:48.126 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803778 -lt 25769803780 2026-02-20T18:05:48.127 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:05:49.127 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:05:49.128 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:05:49.355 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803781 -lt 25769803780 2026-02-20T18:05:49.355 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:05:49.355 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607556 2026-02-20T18:05:49.355 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:05:49.356 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:05:49.357 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-51539607556 2026-02-20T18:05:49.357 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:05:49.358 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=51539607556 2026-02-20T18:05:49.358 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.1 seq 51539607556 2026-02-20T18:05:49.358 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 51539607556' 2026-02-20T18:05:49.358 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:05:49.584 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 51539607556 -lt 51539607556 2026-02-20T18:05:49.585 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:05:49.585 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444035 2026-02-20T18:05:49.585 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:05:49.586 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:05:49.586 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-73014444035 2026-02-20T18:05:49.587 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:05:49.588 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=73014444035 2026-02-20T18:05:49.588 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 73014444035' 2026-02-20T18:05:49.588 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.2 seq 73014444035 2026-02-20T18:05:49.589 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:05:49.817 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 73014444036 -lt 73014444035 2026-02-20T18:05:49.817 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:05:49.818 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182404 2026-02-20T18:05:49.818 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:05:49.818 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:05:49.819 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-107374182404 2026-02-20T18:05:49.819 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:05:49.820 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=107374182404 2026-02-20T18:05:49.820 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 107374182404' 2026-02-20T18:05:49.820 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.3 seq 107374182404 2026-02-20T18:05:49.821 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:05:50.042 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 107374182404 -lt 107374182404 2026-02-20T18:05:50.043 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:05:50.043 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:05:50.043 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:05:50.360 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:05:50.360 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:05:50.360 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:05:50.360 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:05:50.360 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:05:50.360 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:05:50.361 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:05:50.361 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:05:50.579 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:05:50.580 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:05:50.580 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:05:50.580 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:05:50.896 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:05:50.896 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:05:50.897 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:05:50.897 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:328: TEST_rados_get_bad_size_shard_0: local shard_id=0 2026-02-20T18:05:50.897 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:329: TEST_rados_get_bad_size_shard_0: rados_get_data_bad_size td/test-erasure-eio 0 10 2026-02-20T18:05:50.897 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-20T18:05:50.897 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-20T18:05:50.897 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=0 2026-02-20T18:05:50.897 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-20T18:05:50.897 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=10 2026-02-20T18:05:50.897 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-20T18:05:50.898 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=set 2026-02-20T18:05:50.898 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-20T18:05:50.898 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-85808-0-10 2026-02-20T18:05:50.898 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-85808-0-10 2026-02-20T18:05:50.898 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-20T18:05:50.898 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-20T18:05:50.898 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-85808-0-10 2026-02-20T18:05:50.898 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:05:50.898 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-20T18:05:50.898 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:05:50.898 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-20T18:05:50.898 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:05:50.898 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-20T18:05:50.899 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:05:50.899 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-20T18:05:50.899 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-85808-0-10 td/test-erasure-eio/ORIGINAL 2026-02-20T18:05:50.915 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-85808-0-10 td/test-erasure-eio 0 10 set 2026-02-20T18:05:50.915 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-85808-0-10 2026-02-20T18:05:50.915 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-20T18:05:50.915 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-20T18:05:50.915 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-20T18:05:50.915 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=0 2026-02-20T18:05:50.915 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-20T18:05:50.915 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=10 2026-02-20T18:05:50.915 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-20T18:05:50.915 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-20T18:05:50.915 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-20T18:05:50.915 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-85808-0-10 2026-02-20T18:05:50.915 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:05:50.915 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-85808-0-10 2026-02-20T18:05:50.916 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-85808-0-10 2026-02-20T18:05:50.916 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:05:51.144 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T18:05:51.144 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:05:51.144 INFO:tasks.workunit.client.0.trial195.stderr:2' 2026-02-20T18:05:51.144 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-20T18:05:51.144 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-20T18:05:51.144 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-20T18:05:51.144 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=3 2026-02-20T18:05:51.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-20T18:05:51.427 INFO:tasks.workunit.client.0.trial195.stderr:noout is set 2026-02-20T18:05:51.434 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-20T18:05:51.434 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 10 = 0 ']' 2026-02-20T18:05:51.434 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:224: set_size: dd if=/dev/urandom bs=10 count=1 of=td/test-erasure-eio/CORRUPT 2026-02-20T18:05:51.435 INFO:tasks.workunit.client.0.trial195.stderr:1+0 records in 2026-02-20T18:05:51.435 INFO:tasks.workunit.client.0.trial195.stderr:1+0 records out 2026-02-20T18:05:51.435 INFO:tasks.workunit.client.0.trial195.stderr:10 bytes copied, 3.8912e-05 s, 257 kB/s 2026-02-20T18:05:51.435 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 3 obj-size-85808-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:05:51.435 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-20T18:05:51.435 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-20T18:05:51.435 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-20T18:05:51.435 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-20T18:05:51.435 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 3 obj-size-85808-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:05:51.435 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-20T18:05:51.435 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-20T18:05:51.435 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-20T18:05:51.435 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-20T18:05:51.435 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-20T18:05:51.436 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:05:51.436 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:05:51.436 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:05:51.436 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:05:51.436 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:05:51.743 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:05:51.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 3 obj-size-85808-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:05:51.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-20T18:05:51.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-20T18:05:51.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-20T18:05:51.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-20T18:05:51.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/3 2026-02-20T18:05:51.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/3 obj-size-85808-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:05:51.901 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 3 2026-02-20T18:05:51.902 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-20T18:05:51.902 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-20T18:05:51.902 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-20T18:05:51.902 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-20T18:05:51.902 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-20T18:05:51.902 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:05:51.902 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:05:51.902 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:05:51.902 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:05:51.902 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-20T18:05:51.902 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-20T18:05:51.903 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-20T18:05:51.903 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-20T18:05:51.903 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:05:51.903 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-20T18:05:51.903 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:05:51.903 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:05:51.903 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:05:51.903 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:05:51.903 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:05:51.903 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:05:51.904 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:05:51.904 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:05:51.904 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:05:51.904 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:05:51.904 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:05:51.904 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:05:51.904 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:05:51.904 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:05:51.904 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-20T18:05:51.904 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-20T18:05:51.904 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-20T18:05:51.905 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-20T18:05:51.905 INFO:tasks.workunit.client.0.trial195.stderr:start osd.3 2026-02-20T18:05:51.905 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:05:51.905 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-20T18:05:51.906 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-20T18:05:51.906 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-20T18:05:51.906 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-20T18:05:51.906 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-20T18:05:51.918 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:51.916+0000 7fad58453900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:51.919 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:51.917+0000 7fad58453900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:51.920 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:51.918+0000 7fad58453900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:52.046 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:52.044+0000 7fad58453900 -1 Falling back to public interface 2026-02-20T18:05:52.131 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-20T18:05:52.131 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:05:52.131 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-20T18:05:52.131 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:05:52.131 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:05:52.131 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:05:52.131 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:05:52.131 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T18:05:52.132 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:05:52.132 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T18:05:52.358 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:05:52.362 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:52.360+0000 7fad58453900 -1 osd.3 35 log_to_monitors true 2026-02-20T18:05:53.359 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:05:53.359 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:05:53.359 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:05:53.359 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:05:53.359 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:05:53.359 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T18:05:53.584 INFO:tasks.workunit.client.0.trial195.stderr:osd.3 up in weight 1 up_from 39 up_thru 39 down_at 36 last_clean_interval [25,35) [v2:127.0.0.1:6826/4149379347,v1:127.0.0.1:6827/4149379347] [v2:127.0.0.1:6828/4149379347,v1:127.0.0.1:6829/4149379347] exists,up 90b62f9b-4234-48a4-9308-02e49957e131 2026-02-20T18:05:53.584 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:05:53.584 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:05:53.584 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:05:53.584 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-20T18:05:53.584 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:05:53.585 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:05:53.585 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:05:53.585 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:05:53.585 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:05:53.585 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:05:53.585 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:05:53.585 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:05:53.585 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:05:53.672 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:05:53.672 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:05:53.672 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:05:53.672 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:05:53.672 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:05:53.672 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:05:53.895 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:05:53.895 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:05:53.895 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:05:53.896 INFO:tasks.workunit.client.0.trial195.stderr:3' 2026-02-20T18:05:53.896 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:05:53.896 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:05:53.896 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:05:53.962 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803784 2026-02-20T18:05:53.962 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803784 2026-02-20T18:05:53.962 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803784' 2026-02-20T18:05:53.962 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:05:53.962 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:05:54.027 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607559 2026-02-20T18:05:54.028 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607559 2026-02-20T18:05:54.028 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803784 1-51539607559' 2026-02-20T18:05:54.028 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:05:54.028 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:05:54.091 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444039 2026-02-20T18:05:54.091 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444039 2026-02-20T18:05:54.091 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803784 1-51539607559 2-73014444039' 2026-02-20T18:05:54.091 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:05:54.091 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:05:54.157 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724547 2026-02-20T18:05:54.157 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724547 2026-02-20T18:05:54.158 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803784 1-51539607559 2-73014444039 3-167503724547' 2026-02-20T18:05:54.158 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:05:54.158 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803784 2026-02-20T18:05:54.158 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:05:54.159 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:05:54.159 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803784 2026-02-20T18:05:54.159 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:05:54.161 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803784 2026-02-20T18:05:54.161 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-20T18:05:54.161 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.0 seq 25769803784 2026-02-20T18:05:54.161 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:05:54.384 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803781 -lt 25769803784 2026-02-20T18:05:54.384 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:05:55.385 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:05:55.386 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:05:55.610 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803784 -lt 25769803784 2026-02-20T18:05:55.610 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:05:55.610 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607559 2026-02-20T18:05:55.611 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:05:55.611 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:05:55.612 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-51539607559 2026-02-20T18:05:55.612 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:05:55.613 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=51539607559 2026-02-20T18:05:55.613 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 51539607559' 2026-02-20T18:05:55.613 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.1 seq 51539607559 2026-02-20T18:05:55.613 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:05:55.837 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 51539607559 -lt 51539607559 2026-02-20T18:05:55.837 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:05:55.838 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444039 2026-02-20T18:05:55.838 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:05:55.838 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:05:55.839 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-73014444039 2026-02-20T18:05:55.839 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:05:55.840 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=73014444039 2026-02-20T18:05:55.840 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 73014444039' 2026-02-20T18:05:55.840 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.2 seq 73014444039 2026-02-20T18:05:55.840 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:05:56.065 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 73014444039 -lt 73014444039 2026-02-20T18:05:56.065 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:05:56.066 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-167503724547 2026-02-20T18:05:56.066 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:05:56.066 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:05:56.067 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-167503724547 2026-02-20T18:05:56.067 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:05:56.068 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=167503724547 2026-02-20T18:05:56.068 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 167503724547' 2026-02-20T18:05:56.068 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.3 seq 167503724547 2026-02-20T18:05:56.069 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:05:56.291 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 167503724547 -lt 167503724547 2026-02-20T18:05:56.291 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:05:56.292 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:05:56.292 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:05:56.606 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:05:56.606 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:05:56.606 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:05:56.607 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:05:56.607 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:05:56.607 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:05:56.607 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:05:56.607 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:05:56.829 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:05:56.829 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:05:56.830 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:05:56.830 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:05:57.148 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:05:57.149 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:05:57.149 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:05:57.149 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-20T18:05:57.149 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-20T18:05:57.463 INFO:tasks.workunit.client.0.trial195.stderr:noout is unset 2026-02-20T18:05:57.473 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-85808-0-10 2026-02-20T18:05:57.474 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-20T18:05:57.474 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-20T18:05:57.474 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-85808-0-10 2026-02-20T18:05:57.474 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-20T18:05:57.474 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-20T18:05:57.474 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-85808-0-10 td/test-erasure-eio/COPY 2026-02-20T18:05:57.495 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-20T18:05:57.496 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-20T18:05:57.497 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 0 + 1 2026-02-20T18:05:57.499 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=1 2026-02-20T18:05:57.499 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-85808-0-10 td/test-erasure-eio 1 10 set 2026-02-20T18:05:57.499 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-85808-0-10 2026-02-20T18:05:57.499 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-20T18:05:57.499 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-20T18:05:57.499 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-20T18:05:57.499 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-20T18:05:57.499 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-20T18:05:57.499 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=10 2026-02-20T18:05:57.499 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-20T18:05:57.499 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-20T18:05:57.500 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-20T18:05:57.500 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-85808-0-10 2026-02-20T18:05:57.500 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:05:57.500 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-85808-0-10 2026-02-20T18:05:57.500 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-85808-0-10 2026-02-20T18:05:57.500 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:05:57.723 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T18:05:57.724 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:05:57.724 INFO:tasks.workunit.client.0.trial195.stderr:2' 2026-02-20T18:05:57.724 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-20T18:05:57.724 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-20T18:05:57.724 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-20T18:05:57.724 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-20T18:05:57.724 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-20T18:05:58.037 INFO:tasks.workunit.client.0.trial195.stderr:noout is set 2026-02-20T18:05:58.045 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-20T18:05:58.046 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 10 = 0 ']' 2026-02-20T18:05:58.046 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:224: set_size: dd if=/dev/urandom bs=10 count=1 of=td/test-erasure-eio/CORRUPT 2026-02-20T18:05:58.048 INFO:tasks.workunit.client.0.trial195.stderr:1+0 records in 2026-02-20T18:05:58.049 INFO:tasks.workunit.client.0.trial195.stderr:1+0 records out 2026-02-20T18:05:58.049 INFO:tasks.workunit.client.0.trial195.stderr:10 bytes copied, 3.4915e-05 s, 286 kB/s 2026-02-20T18:05:58.049 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-85808-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:05:58.049 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-20T18:05:58.049 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-20T18:05:58.049 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-20T18:05:58.049 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-20T18:05:58.049 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-85808-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:05:58.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-20T18:05:58.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-20T18:05:58.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-20T18:05:58.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-20T18:05:58.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-20T18:05:58.050 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:05:58.050 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:05:58.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:05:58.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:05:58.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:05:58.354 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:05:58.354 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-85808-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:05:58.355 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-20T18:05:58.355 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-20T18:05:58.355 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-20T18:05:58.355 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-20T18:05:58.355 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-20T18:05:58.355 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-85808-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:05:58.500 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-20T18:05:58.500 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-20T18:05:58.500 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-20T18:05:58.501 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-20T18:05:58.501 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-20T18:05:58.501 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-20T18:05:58.501 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:05:58.501 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:05:58.501 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:05:58.501 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:05:58.501 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-20T18:05:58.501 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-20T18:05:58.501 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-20T18:05:58.501 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-20T18:05:58.501 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:05:58.501 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-20T18:05:58.501 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:05:58.501 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:05:58.502 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:05:58.502 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:05:58.502 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:05:58.502 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:05:58.502 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:05:58.502 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:05:58.502 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:05:58.502 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:05:58.503 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:05:58.503 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:05:58.503 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:05:58.503 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:05:58.503 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-20T18:05:58.503 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-20T18:05:58.503 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-20T18:05:58.503 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-20T18:05:58.503 INFO:tasks.workunit.client.0.trial195.stderr:start osd.1 2026-02-20T18:05:58.503 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:05:58.503 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-20T18:05:58.503 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-20T18:05:58.504 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-20T18:05:58.504 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-20T18:05:58.504 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-20T18:05:58.519 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:58.517+0000 7fe1781bf900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:58.520 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:58.519+0000 7fe1781bf900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:58.521 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:58.520+0000 7fe1781bf900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:05:58.671 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:58.669+0000 7fe1781bf900 -1 Falling back to public interface 2026-02-20T18:05:58.732 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-20T18:05:58.733 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:05:58.733 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-20T18:05:58.733 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:05:58.733 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:05:58.733 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:05:58.733 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:05:58.733 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T18:05:58.733 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:05:58.733 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:05:58.957 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:05:58.994 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:05:58.992+0000 7fe1781bf900 -1 osd.1 43 log_to_monitors true 2026-02-20T18:05:59.959 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:05:59.959 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:05:59.959 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:05:59.959 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:05:59.959 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:05:59.960 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:06:00.182 INFO:tasks.workunit.client.0.trial195.stderr:osd.1 up in weight 1 up_from 49 up_thru 49 down_at 46 last_clean_interval [12,43) [v2:127.0.0.1:6810/1871596740,v1:127.0.0.1:6811/1871596740] [v2:127.0.0.1:6812/1871596740,v1:127.0.0.1:6813/1871596740] exists,up cf55ea6b-c46c-423c-b53a-234bec6c36cc 2026-02-20T18:06:00.183 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:06:00.183 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:06:00.183 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:06:00.183 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-20T18:06:00.183 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:06:00.183 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:06:00.183 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:06:00.183 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:06:00.183 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:06:00.184 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:06:00.184 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:06:00.184 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:06:00.184 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:06:00.266 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:06:00.266 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:06:00.267 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:06:00.267 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:06:00.267 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:06:00.267 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:06:00.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:06:00.492 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:06:00.492 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:06:00.492 INFO:tasks.workunit.client.0.trial195.stderr:3' 2026-02-20T18:06:00.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:06:00.493 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:06:00.493 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:06:00.555 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803787 2026-02-20T18:06:00.556 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803787 2026-02-20T18:06:00.556 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803787' 2026-02-20T18:06:00.556 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:06:00.556 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:06:00.623 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=210453397507 2026-02-20T18:06:00.623 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 210453397507 2026-02-20T18:06:00.623 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803787 1-210453397507' 2026-02-20T18:06:00.624 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:06:00.624 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:06:00.693 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444042 2026-02-20T18:06:00.693 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444042 2026-02-20T18:06:00.693 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803787 1-210453397507 2-73014444042' 2026-02-20T18:06:00.693 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:06:00.694 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:06:00.761 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724550 2026-02-20T18:06:00.761 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724550 2026-02-20T18:06:00.761 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803787 1-210453397507 2-73014444042 3-167503724550' 2026-02-20T18:06:00.761 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:06:00.761 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803787 2026-02-20T18:06:00.762 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:06:00.762 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:06:00.763 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803787 2026-02-20T18:06:00.763 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:06:00.764 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803787 2026-02-20T18:06:00.764 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803787' 2026-02-20T18:06:00.764 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.0 seq 25769803787 2026-02-20T18:06:00.764 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:06:00.997 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803785 -lt 25769803787 2026-02-20T18:06:00.997 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:06:01.998 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:06:01.999 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:06:02.226 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803785 -lt 25769803787 2026-02-20T18:06:02.227 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:06:03.228 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-20T18:06:03.228 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:06:03.456 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803787 -lt 25769803787 2026-02-20T18:06:03.456 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:06:03.457 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-210453397507 2026-02-20T18:06:03.457 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:06:03.458 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:06:03.458 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-210453397507 2026-02-20T18:06:03.458 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:06:03.460 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=210453397507 2026-02-20T18:06:03.460 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 210453397507' 2026-02-20T18:06:03.460 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.1 seq 210453397507 2026-02-20T18:06:03.460 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:06:03.683 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 210453397507 -lt 210453397507 2026-02-20T18:06:03.684 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:06:03.684 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444042 2026-02-20T18:06:03.684 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:06:03.685 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:06:03.685 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-73014444042 2026-02-20T18:06:03.686 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:06:03.687 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=73014444042 2026-02-20T18:06:03.687 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 73014444042' 2026-02-20T18:06:03.687 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.2 seq 73014444042 2026-02-20T18:06:03.687 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:06:03.910 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 73014444042 -lt 73014444042 2026-02-20T18:06:03.911 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:06:03.911 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-167503724550 2026-02-20T18:06:03.912 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:06:03.912 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:06:03.913 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-167503724550 2026-02-20T18:06:03.913 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:06:03.914 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=167503724550 2026-02-20T18:06:03.914 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 167503724550' 2026-02-20T18:06:03.914 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.3 seq 167503724550 2026-02-20T18:06:03.914 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:06:04.140 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 167503724551 -lt 167503724550 2026-02-20T18:06:04.140 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:06:04.140 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:06:04.140 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:06:04.458 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:06:04.458 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:06:04.458 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:06:04.458 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:06:04.458 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:06:04.458 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:06:04.459 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:06:04.459 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:06:04.681 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:06:04.681 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:06:04.682 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:06:04.682 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:06:04.992 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:06:04.992 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:06:04.992 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:06:04.992 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-20T18:06:04.993 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-20T18:06:05.305 INFO:tasks.workunit.client.0.trial195.stderr:noout is unset 2026-02-20T18:06:05.316 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 0 -eq 1 -a 1 -eq 2 ']' 2026-02-20T18:06:05.316 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:262: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-85808-0-10 fail 2026-02-20T18:06:05.316 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-20T18:06:05.317 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-20T18:06:05.317 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-85808-0-10 2026-02-20T18:06:05.317 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-20T18:06:05.317 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-20T18:06:05.317 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-size-85808-0-10 td/test-erasure-eio/COPY 2026-02-20T18:06:05.336 INFO:tasks.workunit.client.0.trial195.stderr:error getting pool-jerasure/obj-size-85808-0-10: (5) Input/output error 2026-02-20T18:06:05.337 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-20T18:06:05.337 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-20T18:06:05.340 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:330: TEST_rados_get_bad_size_shard_0: rados_get_data_bad_size td/test-erasure-eio 0 0 2026-02-20T18:06:05.340 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-20T18:06:05.340 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-20T18:06:05.341 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=0 2026-02-20T18:06:05.341 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-20T18:06:05.341 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=0 2026-02-20T18:06:05.341 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-20T18:06:05.341 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=set 2026-02-20T18:06:05.341 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-20T18:06:05.341 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-85808-0-0 2026-02-20T18:06:05.341 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-85808-0-0 2026-02-20T18:06:05.341 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-20T18:06:05.341 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-20T18:06:05.341 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-85808-0-0 2026-02-20T18:06:05.342 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:06:05.342 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-20T18:06:05.342 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:06:05.342 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-20T18:06:05.342 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:06:05.342 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-20T18:06:05.342 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:06:05.342 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-20T18:06:05.342 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-85808-0-0 td/test-erasure-eio/ORIGINAL 2026-02-20T18:06:05.359 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-85808-0-0 td/test-erasure-eio 0 0 set 2026-02-20T18:06:05.359 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-85808-0-0 2026-02-20T18:06:05.359 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-20T18:06:05.359 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-20T18:06:05.359 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-20T18:06:05.359 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=0 2026-02-20T18:06:05.359 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-20T18:06:05.360 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=0 2026-02-20T18:06:05.360 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-20T18:06:05.360 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-20T18:06:05.360 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-20T18:06:05.360 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-85808-0-0 2026-02-20T18:06:05.360 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:06:05.360 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-85808-0-0 2026-02-20T18:06:05.360 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-85808-0-0 2026-02-20T18:06:05.360 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:06:05.589 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T18:06:05.589 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:06:05.589 INFO:tasks.workunit.client.0.trial195.stderr:2' 2026-02-20T18:06:05.589 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-20T18:06:05.589 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-20T18:06:05.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-20T18:06:05.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=3 2026-02-20T18:06:05.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-20T18:06:05.872 INFO:tasks.workunit.client.0.trial195.stderr:noout is set 2026-02-20T18:06:05.880 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-20T18:06:05.880 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 0 = 0 ']' 2026-02-20T18:06:05.880 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:222: set_size: touch td/test-erasure-eio/CORRUPT 2026-02-20T18:06:05.882 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 3 obj-size-85808-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:06:05.882 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-20T18:06:05.882 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-20T18:06:05.883 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-20T18:06:05.883 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-20T18:06:05.883 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 3 obj-size-85808-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:06:05.883 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-20T18:06:05.883 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-20T18:06:05.883 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-20T18:06:05.883 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-20T18:06:05.883 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-20T18:06:05.883 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:06:05.883 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:06:05.883 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:06:05.883 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:06:05.884 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:06:06.189 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:06:06.189 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 3 obj-size-85808-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:06:06.190 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-20T18:06:06.190 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-20T18:06:06.190 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-20T18:06:06.190 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-20T18:06:06.190 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/3 2026-02-20T18:06:06.190 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/3 obj-size-85808-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:06:06.325 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 3 2026-02-20T18:06:06.326 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-20T18:06:06.326 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-20T18:06:06.326 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-20T18:06:06.326 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-20T18:06:06.326 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-20T18:06:06.326 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:06:06.326 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:06:06.326 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:06:06.326 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:06:06.327 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-20T18:06:06.327 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-20T18:06:06.327 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-20T18:06:06.327 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-20T18:06:06.327 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:06:06.327 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-20T18:06:06.327 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:06:06.327 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:06:06.327 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:06:06.327 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:06:06.327 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:06:06.327 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:06:06.328 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:06:06.328 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:06:06.328 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:06:06.328 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:06:06.328 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:06:06.328 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:06:06.328 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:06:06.328 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:06:06.328 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-20T18:06:06.328 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-20T18:06:06.328 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-20T18:06:06.329 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-20T18:06:06.329 INFO:tasks.workunit.client.0.trial195.stderr:start osd.3 2026-02-20T18:06:06.329 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:06:06.329 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-20T18:06:06.330 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-20T18:06:06.330 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-20T18:06:06.330 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-20T18:06:06.330 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-20T18:06:06.342 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:06.340+0000 7f49a6d3b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:06.343 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:06.341+0000 7f49a6d3b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:06.344 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:06.342+0000 7f49a6d3b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:06.484 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:06.482+0000 7f49a6d3b900 -1 Falling back to public interface 2026-02-20T18:06:06.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-20T18:06:06.555 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:06:06.555 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-20T18:06:06.555 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:06:06.555 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:06:06.555 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:06:06.555 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:06:06.555 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T18:06:06.555 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:06:06.555 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T18:06:06.778 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:06:06.831 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:06.829+0000 7f49a6d3b900 -1 osd.3 54 log_to_monitors true 2026-02-20T18:06:07.779 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:06:07.780 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:06:07.780 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:06:07.780 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:06:07.780 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:06:07.780 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T18:06:08.006 INFO:tasks.workunit.client.0.trial195.stderr:osd.3 up in weight 1 up_from 59 up_thru 59 down_at 56 last_clean_interval [39,54) [v2:127.0.0.1:6826/4195560782,v1:127.0.0.1:6827/4195560782] [v2:127.0.0.1:6828/4195560782,v1:127.0.0.1:6829/4195560782] exists,up 90b62f9b-4234-48a4-9308-02e49957e131 2026-02-20T18:06:08.006 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:06:08.007 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:06:08.007 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:06:08.007 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-20T18:06:08.007 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:06:08.007 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:06:08.007 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:06:08.007 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:06:08.008 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:06:08.008 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:06:08.008 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:06:08.008 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:06:08.008 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:06:08.095 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:06:08.095 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:06:08.095 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:06:08.095 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:06:08.095 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:06:08.095 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:06:08.318 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:06:08.318 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:06:08.318 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:06:08.318 INFO:tasks.workunit.client.0.trial195.stderr:3' 2026-02-20T18:06:08.318 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:06:08.318 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:06:08.319 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:06:08.383 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803791 2026-02-20T18:06:08.383 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803791 2026-02-20T18:06:08.383 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803791' 2026-02-20T18:06:08.383 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:06:08.383 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:06:08.448 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=210453397510 2026-02-20T18:06:08.448 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 210453397510 2026-02-20T18:06:08.448 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803791 1-210453397510' 2026-02-20T18:06:08.448 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:06:08.448 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:06:08.512 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444046 2026-02-20T18:06:08.513 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444046 2026-02-20T18:06:08.513 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803791 1-210453397510 2-73014444046' 2026-02-20T18:06:08.513 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:06:08.513 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:06:08.577 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=253403070467 2026-02-20T18:06:08.577 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 253403070467 2026-02-20T18:06:08.577 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803791 1-210453397510 2-73014444046 3-253403070467' 2026-02-20T18:06:08.578 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:06:08.578 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803791 2026-02-20T18:06:08.578 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:06:08.579 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:06:08.579 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803791 2026-02-20T18:06:08.579 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:06:08.580 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803791 2026-02-20T18:06:08.580 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803791' 2026-02-20T18:06:08.581 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.0 seq 25769803791 2026-02-20T18:06:08.581 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:06:08.806 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803791 -lt 25769803791 2026-02-20T18:06:08.806 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:06:08.807 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-210453397510 2026-02-20T18:06:08.807 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:06:08.807 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:06:08.808 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-210453397510 2026-02-20T18:06:08.808 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:06:08.809 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=210453397510 2026-02-20T18:06:08.809 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 210453397510' 2026-02-20T18:06:08.809 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.1 seq 210453397510 2026-02-20T18:06:08.810 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:06:09.037 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 210453397510 -lt 210453397510 2026-02-20T18:06:09.037 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:06:09.038 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444046 2026-02-20T18:06:09.038 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:06:09.038 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:06:09.039 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-73014444046 2026-02-20T18:06:09.039 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:06:09.040 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=73014444046 2026-02-20T18:06:09.040 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 73014444046' 2026-02-20T18:06:09.041 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.2 seq 73014444046 2026-02-20T18:06:09.041 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:06:09.265 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 73014444046 -lt 73014444046 2026-02-20T18:06:09.265 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:06:09.266 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-253403070467 2026-02-20T18:06:09.266 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:06:09.266 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:06:09.267 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-253403070467 2026-02-20T18:06:09.267 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:06:09.268 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=253403070467 2026-02-20T18:06:09.268 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 253403070467' 2026-02-20T18:06:09.268 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.3 seq 253403070467 2026-02-20T18:06:09.269 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:06:09.497 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 253403070467 2026-02-20T18:06:09.497 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:06:10.498 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:06:10.498 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:06:10.729 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 253403070467 -lt 253403070467 2026-02-20T18:06:10.730 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:06:10.730 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:06:10.730 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:06:11.057 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:06:11.057 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:06:11.058 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:06:11.058 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:06:11.058 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:06:11.058 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:06:11.058 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:06:11.058 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:06:11.282 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:06:11.283 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:06:11.283 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:06:11.283 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:06:11.591 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:06:11.592 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:06:11.592 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:06:11.592 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-20T18:06:11.593 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-20T18:06:11.908 INFO:tasks.workunit.client.0.trial195.stderr:noout is unset 2026-02-20T18:06:11.917 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-85808-0-0 2026-02-20T18:06:11.917 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-20T18:06:11.917 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-20T18:06:11.917 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-85808-0-0 2026-02-20T18:06:11.917 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-20T18:06:11.917 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-20T18:06:11.917 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-85808-0-0 td/test-erasure-eio/COPY 2026-02-20T18:06:11.938 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-20T18:06:11.939 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-20T18:06:11.940 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 0 + 1 2026-02-20T18:06:11.942 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=1 2026-02-20T18:06:11.942 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-85808-0-0 td/test-erasure-eio 1 0 set 2026-02-20T18:06:11.942 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-85808-0-0 2026-02-20T18:06:11.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-20T18:06:11.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-20T18:06:11.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-20T18:06:11.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-20T18:06:11.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-20T18:06:11.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=0 2026-02-20T18:06:11.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-20T18:06:11.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-20T18:06:11.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-20T18:06:11.943 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-85808-0-0 2026-02-20T18:06:11.943 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:06:11.943 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-85808-0-0 2026-02-20T18:06:11.944 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-85808-0-0 2026-02-20T18:06:11.944 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:06:12.171 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T18:06:12.171 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:06:12.171 INFO:tasks.workunit.client.0.trial195.stderr:2' 2026-02-20T18:06:12.171 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-20T18:06:12.171 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-20T18:06:12.171 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-20T18:06:12.172 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-20T18:06:12.172 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-20T18:06:12.486 INFO:tasks.workunit.client.0.trial195.stderr:noout is set 2026-02-20T18:06:12.496 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-20T18:06:12.496 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 0 = 0 ']' 2026-02-20T18:06:12.497 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:222: set_size: touch td/test-erasure-eio/CORRUPT 2026-02-20T18:06:12.498 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-85808-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:06:12.498 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-20T18:06:12.499 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-20T18:06:12.499 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-20T18:06:12.499 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-20T18:06:12.499 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-85808-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:06:12.499 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-20T18:06:12.499 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-20T18:06:12.499 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-20T18:06:12.499 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-20T18:06:12.499 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-20T18:06:12.499 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:06:12.499 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:06:12.500 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:06:12.500 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:06:12.500 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:06:12.806 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:06:12.806 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-85808-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:06:12.807 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-20T18:06:12.807 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-20T18:06:12.807 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-20T18:06:12.807 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-20T18:06:12.807 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-20T18:06:12.807 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-85808-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:06:12.937 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-20T18:06:12.937 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-20T18:06:12.937 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-20T18:06:12.937 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-20T18:06:12.937 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-20T18:06:12.937 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-20T18:06:12.937 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:06:12.938 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:06:12.938 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:06:12.938 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:06:12.938 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-20T18:06:12.938 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-20T18:06:12.938 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-20T18:06:12.938 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-20T18:06:12.938 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:06:12.938 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-20T18:06:12.938 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:06:12.938 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:06:12.939 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:06:12.939 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:06:12.939 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:06:12.939 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:06:12.939 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:06:12.939 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:06:12.939 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:06:12.939 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:06:12.939 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:06:12.939 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:06:12.939 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:06:12.939 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:06:12.939 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-20T18:06:12.940 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-20T18:06:12.940 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-20T18:06:12.940 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-20T18:06:12.940 INFO:tasks.workunit.client.0.trial195.stderr:start osd.1 2026-02-20T18:06:12.940 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:06:12.940 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-20T18:06:12.941 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-20T18:06:12.941 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-20T18:06:12.941 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-20T18:06:12.941 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-20T18:06:12.953 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:12.952+0000 7f5947b88900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:12.954 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:12.953+0000 7f5947b88900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:12.955 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:12.954+0000 7f5947b88900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:13.084 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:13.083+0000 7f5947b88900 -1 Falling back to public interface 2026-02-20T18:06:13.164 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-20T18:06:13.164 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:06:13.164 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-20T18:06:13.164 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:06:13.164 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:06:13.164 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:06:13.164 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:06:13.164 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T18:06:13.165 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:06:13.165 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:06:13.392 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:06:13.408 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:13.408+0000 7f5947b88900 -1 osd.1 62 log_to_monitors true 2026-02-20T18:06:14.393 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:06:14.393 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:06:14.393 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:06:14.394 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:06:14.394 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:06:14.394 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:06:14.619 INFO:tasks.workunit.client.0.trial195.stderr:osd.1 up in weight 1 up_from 68 up_thru 68 down_at 65 last_clean_interval [49,62) [v2:127.0.0.1:6810/1539152144,v1:127.0.0.1:6811/1539152144] [v2:127.0.0.1:6812/1539152144,v1:127.0.0.1:6813/1539152144] exists,up cf55ea6b-c46c-423c-b53a-234bec6c36cc 2026-02-20T18:06:14.619 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:06:14.619 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:06:14.619 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:06:14.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-20T18:06:14.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:06:14.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:06:14.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:06:14.620 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:06:14.620 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:06:14.620 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:06:14.620 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:06:14.620 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:06:14.620 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:06:14.705 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:06:14.705 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:06:14.705 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:06:14.705 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:06:14.705 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:06:14.705 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:06:14.931 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:06:14.931 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:06:14.931 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:06:14.931 INFO:tasks.workunit.client.0.trial195.stderr:3' 2026-02-20T18:06:14.932 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:06:14.932 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:06:14.932 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:06:14.999 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803794 2026-02-20T18:06:14.999 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803794 2026-02-20T18:06:14.999 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803794' 2026-02-20T18:06:14.999 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:06:14.999 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:06:15.063 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=292057776131 2026-02-20T18:06:15.063 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 292057776131 2026-02-20T18:06:15.063 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803794 1-292057776131' 2026-02-20T18:06:15.063 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:06:15.063 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:06:15.130 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444049 2026-02-20T18:06:15.130 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444049 2026-02-20T18:06:15.131 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803794 1-292057776131 2-73014444049' 2026-02-20T18:06:15.131 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:06:15.131 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:06:15.196 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=253403070470 2026-02-20T18:06:15.197 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 253403070470 2026-02-20T18:06:15.197 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803794 1-292057776131 2-73014444049 3-253403070470' 2026-02-20T18:06:15.197 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:06:15.197 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803794 2026-02-20T18:06:15.197 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:06:15.197 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:06:15.198 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803794 2026-02-20T18:06:15.198 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:06:15.199 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803794 2026-02-20T18:06:15.200 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803794' 2026-02-20T18:06:15.200 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.0 seq 25769803794 2026-02-20T18:06:15.200 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:06:15.421 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803792 -lt 25769803794 2026-02-20T18:06:15.422 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:06:16.422 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:06:16.423 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:06:16.650 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803794 -lt 25769803794 2026-02-20T18:06:16.650 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:06:16.650 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-292057776131 2026-02-20T18:06:16.650 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:06:16.651 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:06:16.652 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-292057776131 2026-02-20T18:06:16.652 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:06:16.653 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=292057776131 2026-02-20T18:06:16.653 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 292057776131' 2026-02-20T18:06:16.653 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.1 seq 292057776131 2026-02-20T18:06:16.653 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:06:16.880 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 292057776131 -lt 292057776131 2026-02-20T18:06:16.880 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:06:16.881 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444049 2026-02-20T18:06:16.881 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:06:16.881 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:06:16.882 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-73014444049 2026-02-20T18:06:16.882 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:06:16.883 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=73014444049 2026-02-20T18:06:16.883 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 73014444049' 2026-02-20T18:06:16.883 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.2 seq 73014444049 2026-02-20T18:06:16.884 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:06:17.110 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 73014444049 -lt 73014444049 2026-02-20T18:06:17.110 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:06:17.111 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-253403070470 2026-02-20T18:06:17.111 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:06:17.112 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:06:17.113 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-253403070470 2026-02-20T18:06:17.113 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:06:17.114 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=253403070470 2026-02-20T18:06:17.114 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 253403070470' 2026-02-20T18:06:17.114 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.3 seq 253403070470 2026-02-20T18:06:17.114 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:06:17.337 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 253403070470 -lt 253403070470 2026-02-20T18:06:17.337 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:06:17.337 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:06:17.337 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:06:17.650 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:06:17.651 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:06:17.651 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:06:17.651 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:06:17.651 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:06:17.651 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:06:17.651 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:06:17.651 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:06:17.875 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:06:17.876 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:06:17.876 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:06:17.876 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:06:18.198 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:06:18.198 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:06:18.198 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:06:18.199 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-20T18:06:18.199 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-20T18:06:18.516 INFO:tasks.workunit.client.0.trial195.stderr:noout is unset 2026-02-20T18:06:18.527 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 0 -eq 1 -a 1 -eq 2 ']' 2026-02-20T18:06:18.527 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:262: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-85808-0-0 fail 2026-02-20T18:06:18.527 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-20T18:06:18.527 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-20T18:06:18.527 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-85808-0-0 2026-02-20T18:06:18.527 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-20T18:06:18.528 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-20T18:06:18.528 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-size-85808-0-0 td/test-erasure-eio/COPY 2026-02-20T18:06:18.547 INFO:tasks.workunit.client.0.trial195.stderr:error getting pool-jerasure/obj-size-85808-0-0: (5) Input/output error 2026-02-20T18:06:18.549 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-20T18:06:18.549 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-20T18:06:18.552 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:331: TEST_rados_get_bad_size_shard_0: rados_get_data_bad_size td/test-erasure-eio 0 256 add 2026-02-20T18:06:18.552 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-20T18:06:18.553 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-20T18:06:18.553 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=0 2026-02-20T18:06:18.553 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-20T18:06:18.553 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=256 2026-02-20T18:06:18.553 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-20T18:06:18.553 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=add 2026-02-20T18:06:18.553 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-20T18:06:18.553 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-85808-0-256 2026-02-20T18:06:18.553 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-85808-0-256 2026-02-20T18:06:18.553 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-20T18:06:18.553 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-20T18:06:18.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-85808-0-256 2026-02-20T18:06:18.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:06:18.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-20T18:06:18.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:06:18.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-20T18:06:18.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:06:18.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-20T18:06:18.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:06:18.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-20T18:06:18.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-85808-0-256 td/test-erasure-eio/ORIGINAL 2026-02-20T18:06:18.571 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-85808-0-256 td/test-erasure-eio 0 256 add 2026-02-20T18:06:18.571 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-85808-0-256 2026-02-20T18:06:18.571 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-20T18:06:18.571 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-20T18:06:18.571 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-20T18:06:18.571 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=0 2026-02-20T18:06:18.572 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-20T18:06:18.572 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=256 2026-02-20T18:06:18.572 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-20T18:06:18.572 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=add 2026-02-20T18:06:18.572 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-20T18:06:18.572 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-85808-0-256 2026-02-20T18:06:18.572 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:06:18.573 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-85808-0-256 2026-02-20T18:06:18.573 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-85808-0-256 2026-02-20T18:06:18.573 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:06:18.804 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T18:06:18.804 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:06:18.804 INFO:tasks.workunit.client.0.trial195.stderr:2' 2026-02-20T18:06:18.804 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-20T18:06:18.805 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-20T18:06:18.805 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-20T18:06:18.805 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=3 2026-02-20T18:06:18.805 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-20T18:06:19.086 INFO:tasks.workunit.client.0.trial195.stderr:noout is set 2026-02-20T18:06:19.095 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' add = add ']' 2026-02-20T18:06:19.095 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: objectstore_tool td/test-erasure-eio 3 obj-size-85808-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:06:19.095 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-20T18:06:19.095 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-20T18:06:19.096 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-20T18:06:19.096 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-20T18:06:19.096 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 3 obj-size-85808-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:06:19.096 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-20T18:06:19.096 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-20T18:06:19.096 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-20T18:06:19.096 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-20T18:06:19.096 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-20T18:06:19.096 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:06:19.096 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:06:19.096 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:06:19.096 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:06:19.096 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:06:19.404 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:06:19.405 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 3 obj-size-85808-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:06:19.405 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-20T18:06:19.405 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-20T18:06:19.405 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-20T18:06:19.405 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-20T18:06:19.405 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/3 2026-02-20T18:06:19.405 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/3 obj-size-85808-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:06:19.498 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 3 2026-02-20T18:06:19.498 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-20T18:06:19.498 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-20T18:06:19.498 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-20T18:06:19.498 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-20T18:06:19.498 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-20T18:06:19.498 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:06:19.499 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:06:19.499 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:06:19.499 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:06:19.499 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-20T18:06:19.499 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-20T18:06:19.499 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-20T18:06:19.499 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-20T18:06:19.499 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:06:19.499 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-20T18:06:19.499 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:06:19.499 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:06:19.500 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:06:19.500 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:06:19.500 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:06:19.500 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:06:19.500 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:06:19.500 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:06:19.500 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:06:19.500 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:06:19.500 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:06:19.500 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:06:19.500 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:06:19.500 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:06:19.501 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-20T18:06:19.501 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-20T18:06:19.501 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-20T18:06:19.501 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-20T18:06:19.501 INFO:tasks.workunit.client.0.trial195.stderr:start osd.3 2026-02-20T18:06:19.501 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:06:19.501 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-20T18:06:19.502 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-20T18:06:19.502 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-20T18:06:19.502 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-20T18:06:19.502 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-20T18:06:19.514 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:19.513+0000 7fce16197900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:19.515 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:19.515+0000 7fce16197900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:19.516 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:19.516+0000 7fce16197900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:19.642 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:19.641+0000 7fce16197900 -1 Falling back to public interface 2026-02-20T18:06:19.723 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-20T18:06:19.724 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:06:19.724 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-20T18:06:19.724 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:06:19.724 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:06:19.724 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:06:19.724 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:06:19.724 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T18:06:19.724 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:06:19.724 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T18:06:19.948 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:19.947+0000 7fce16197900 -1 osd.3 71 log_to_monitors true 2026-02-20T18:06:19.952 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:06:20.953 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:06:20.953 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:06:20.953 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:06:20.953 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:06:20.954 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:06:20.954 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T18:06:21.178 INFO:tasks.workunit.client.0.trial195.stderr:osd.3 up in weight 1 up_from 77 up_thru 77 down_at 74 last_clean_interval [59,71) [v2:127.0.0.1:6826/1130349679,v1:127.0.0.1:6827/1130349679] [v2:127.0.0.1:6828/1130349679,v1:127.0.0.1:6829/1130349679] exists,up 90b62f9b-4234-48a4-9308-02e49957e131 2026-02-20T18:06:21.178 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:06:21.178 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:06:21.179 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:06:21.179 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-20T18:06:21.179 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:06:21.179 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:06:21.179 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:06:21.179 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:06:21.179 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:06:21.179 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:06:21.179 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:06:21.179 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:06:21.179 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:06:21.264 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:06:21.264 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:06:21.265 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:06:21.265 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:06:21.265 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:06:21.265 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:06:21.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:06:21.491 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:06:21.491 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:06:21.492 INFO:tasks.workunit.client.0.trial195.stderr:3' 2026-02-20T18:06:21.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:06:21.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:06:21.492 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:06:21.555 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803797 2026-02-20T18:06:21.555 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803797 2026-02-20T18:06:21.555 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803797' 2026-02-20T18:06:21.555 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:06:21.555 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:06:21.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=292057776134 2026-02-20T18:06:21.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 292057776134 2026-02-20T18:06:21.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803797 1-292057776134' 2026-02-20T18:06:21.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:06:21.620 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:06:21.684 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444052 2026-02-20T18:06:21.684 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444052 2026-02-20T18:06:21.684 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803797 1-292057776134 2-73014444052' 2026-02-20T18:06:21.684 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:06:21.685 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:06:21.749 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=330712481795 2026-02-20T18:06:21.749 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 330712481795 2026-02-20T18:06:21.750 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803797 1-292057776134 2-73014444052 3-330712481795' 2026-02-20T18:06:21.750 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:06:21.750 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803797 2026-02-20T18:06:21.750 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:06:21.750 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:06:21.751 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803797 2026-02-20T18:06:21.751 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:06:21.752 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803797 2026-02-20T18:06:21.752 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803797' 2026-02-20T18:06:21.752 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.0 seq 25769803797 2026-02-20T18:06:21.753 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:06:21.978 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803795 -lt 25769803797 2026-02-20T18:06:21.979 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:06:22.979 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:06:22.980 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:06:23.201 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803797 -lt 25769803797 2026-02-20T18:06:23.201 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:06:23.201 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-292057776134 2026-02-20T18:06:23.201 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:06:23.202 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:06:23.203 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-292057776134 2026-02-20T18:06:23.203 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:06:23.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=292057776134 2026-02-20T18:06:23.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 292057776134' 2026-02-20T18:06:23.204 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.1 seq 292057776134 2026-02-20T18:06:23.205 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:06:23.431 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 292057776134 -lt 292057776134 2026-02-20T18:06:23.431 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:06:23.431 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444052 2026-02-20T18:06:23.432 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:06:23.433 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:06:23.433 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-73014444052 2026-02-20T18:06:23.433 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:06:23.435 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=73014444052 2026-02-20T18:06:23.435 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 73014444052' 2026-02-20T18:06:23.435 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.2 seq 73014444052 2026-02-20T18:06:23.435 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:06:23.666 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 73014444052 -lt 73014444052 2026-02-20T18:06:23.666 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:06:23.666 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-330712481795 2026-02-20T18:06:23.667 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:06:23.668 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:06:23.668 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-330712481795 2026-02-20T18:06:23.668 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:06:23.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=330712481795 2026-02-20T18:06:23.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 330712481795' 2026-02-20T18:06:23.670 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.3 seq 330712481795 2026-02-20T18:06:23.670 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:06:23.891 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 330712481795 -lt 330712481795 2026-02-20T18:06:23.892 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:06:23.892 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:06:23.892 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:06:24.203 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:06:24.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:06:24.204 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:06:24.204 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:06:24.204 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:06:24.204 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:06:24.204 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:06:24.204 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:06:24.427 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:06:24.428 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:06:24.428 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:06:24.428 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:06:24.747 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:06:24.747 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:06:24.748 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:06:24.748 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:219: set_size: dd if=/dev/urandom bs=256 count=1 2026-02-20T18:06:24.749 INFO:tasks.workunit.client.0.trial195.stderr:1+0 records in 2026-02-20T18:06:24.750 INFO:tasks.workunit.client.0.trial195.stderr:1+0 records out 2026-02-20T18:06:24.750 INFO:tasks.workunit.client.0.trial195.stderr:256 bytes copied, 2.5998e-05 s, 9.8 MB/s 2026-02-20T18:06:24.750 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 3 obj-size-85808-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:06:24.750 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-20T18:06:24.750 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-20T18:06:24.750 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-20T18:06:24.750 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-20T18:06:24.750 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 3 obj-size-85808-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:06:24.750 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-20T18:06:24.750 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-20T18:06:24.751 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-20T18:06:24.751 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-20T18:06:24.751 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-20T18:06:24.751 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:06:24.751 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:06:24.751 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:06:24.751 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:06:24.751 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:06:24.855 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:06:24.855 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 3 obj-size-85808-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:06:24.855 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-20T18:06:24.855 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-20T18:06:24.855 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-20T18:06:24.855 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-20T18:06:24.855 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/3 2026-02-20T18:06:24.855 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/3 obj-size-85808-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:06:25.011 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 3 2026-02-20T18:06:25.011 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-20T18:06:25.011 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-20T18:06:25.012 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-20T18:06:25.012 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-20T18:06:25.012 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-20T18:06:25.012 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:06:25.012 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:06:25.012 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:06:25.012 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:06:25.013 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-20T18:06:25.013 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-20T18:06:25.013 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-20T18:06:25.013 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-20T18:06:25.013 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:06:25.013 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-20T18:06:25.013 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:06:25.013 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:06:25.013 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:06:25.013 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:06:25.013 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:06:25.013 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:06:25.014 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:06:25.014 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:06:25.014 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:06:25.014 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:06:25.014 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:06:25.014 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:06:25.014 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:06:25.015 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:06:25.015 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-20T18:06:25.015 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-20T18:06:25.015 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-20T18:06:25.015 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-20T18:06:25.015 INFO:tasks.workunit.client.0.trial195.stderr:start osd.3 2026-02-20T18:06:25.015 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:06:25.015 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-20T18:06:25.016 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-20T18:06:25.017 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-20T18:06:25.017 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-20T18:06:25.017 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-20T18:06:25.027 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:25.026+0000 7fd89d975900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:25.028 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:25.027+0000 7fd89d975900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:25.029 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:25.028+0000 7fd89d975900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:25.221 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:25.220+0000 7fd89d975900 -1 Falling back to public interface 2026-02-20T18:06:25.240 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-20T18:06:25.240 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:06:25.240 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-20T18:06:25.240 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:06:25.240 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:06:25.240 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:06:25.240 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:06:25.240 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T18:06:25.241 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:06:25.241 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T18:06:25.466 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:06:25.575 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:25.574+0000 7fd89d975900 -1 osd.3 79 log_to_monitors true 2026-02-20T18:06:26.468 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:06:26.468 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:06:26.468 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:06:26.468 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:06:26.468 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:06:26.469 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T18:06:26.695 INFO:tasks.workunit.client.0.trial195.stderr:osd.3 up in weight 1 up_from 83 up_thru 83 down_at 80 last_clean_interval [77,79) [v2:127.0.0.1:6826/4180867950,v1:127.0.0.1:6827/4180867950] [v2:127.0.0.1:6828/4180867950,v1:127.0.0.1:6829/4180867950] exists,up 90b62f9b-4234-48a4-9308-02e49957e131 2026-02-20T18:06:26.695 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:06:26.695 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:06:26.695 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:06:26.695 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-20T18:06:26.695 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:06:26.696 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:06:26.696 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:06:26.696 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:06:26.696 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:06:26.696 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:06:26.696 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:06:26.696 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:06:26.696 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:06:26.782 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:06:26.782 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:06:26.783 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:06:26.783 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:06:26.783 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:06:26.783 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:06:27.005 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:06:27.006 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:06:27.006 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:06:27.006 INFO:tasks.workunit.client.0.trial195.stderr:3' 2026-02-20T18:06:27.006 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:06:27.007 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:06:27.007 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:06:27.070 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803800 2026-02-20T18:06:27.070 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803800 2026-02-20T18:06:27.071 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803800' 2026-02-20T18:06:27.071 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:06:27.071 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:06:27.135 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=292057776137 2026-02-20T18:06:27.135 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 292057776137 2026-02-20T18:06:27.135 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803800 1-292057776137' 2026-02-20T18:06:27.135 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:06:27.135 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:06:27.199 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444055 2026-02-20T18:06:27.200 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444055 2026-02-20T18:06:27.200 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803800 1-292057776137 2-73014444055' 2026-02-20T18:06:27.200 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:06:27.200 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:06:27.265 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=356482285571 2026-02-20T18:06:27.265 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 356482285571 2026-02-20T18:06:27.265 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803800 1-292057776137 2-73014444055 3-356482285571' 2026-02-20T18:06:27.266 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:06:27.266 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803800 2026-02-20T18:06:27.266 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:06:27.266 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:06:27.267 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803800 2026-02-20T18:06:27.267 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:06:27.268 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803800 2026-02-20T18:06:27.268 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803800' 2026-02-20T18:06:27.268 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.0 seq 25769803800 2026-02-20T18:06:27.269 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:06:27.494 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803798 -lt 25769803800 2026-02-20T18:06:27.494 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:06:28.495 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:06:28.496 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:06:28.723 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803801 -lt 25769803800 2026-02-20T18:06:28.723 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:06:28.724 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-292057776137 2026-02-20T18:06:28.724 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:06:28.724 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:06:28.725 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-292057776137 2026-02-20T18:06:28.725 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:06:28.726 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=292057776137 2026-02-20T18:06:28.726 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 292057776137' 2026-02-20T18:06:28.726 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.1 seq 292057776137 2026-02-20T18:06:28.727 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:06:28.953 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 292057776138 -lt 292057776137 2026-02-20T18:06:28.953 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:06:28.953 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444055 2026-02-20T18:06:28.953 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:06:28.954 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:06:28.955 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-73014444055 2026-02-20T18:06:28.955 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:06:28.956 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=73014444055 2026-02-20T18:06:28.956 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 73014444055' 2026-02-20T18:06:28.956 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.2 seq 73014444055 2026-02-20T18:06:28.956 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:06:29.179 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 73014444056 -lt 73014444055 2026-02-20T18:06:29.179 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:06:29.179 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-356482285571 2026-02-20T18:06:29.179 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:06:29.180 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:06:29.181 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-356482285571 2026-02-20T18:06:29.181 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:06:29.182 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=356482285571 2026-02-20T18:06:29.182 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 356482285571' 2026-02-20T18:06:29.182 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.3 seq 356482285571 2026-02-20T18:06:29.182 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:06:29.412 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 356482285571 -lt 356482285571 2026-02-20T18:06:29.413 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:06:29.413 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:06:29.413 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:06:29.724 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:06:29.725 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:06:29.725 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:06:29.725 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:06:29.725 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:06:29.725 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:06:29.725 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:06:29.725 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:06:29.954 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:06:29.955 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:06:29.955 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:06:29.955 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:06:30.272 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:06:30.272 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:06:30.272 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:06:30.272 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-20T18:06:30.273 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-20T18:06:30.586 INFO:tasks.workunit.client.0.trial195.stderr:noout is unset 2026-02-20T18:06:30.595 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-85808-0-256 2026-02-20T18:06:30.596 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-20T18:06:30.596 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-20T18:06:30.596 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-85808-0-256 2026-02-20T18:06:30.596 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-20T18:06:30.596 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-20T18:06:30.596 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-85808-0-256 td/test-erasure-eio/COPY 2026-02-20T18:06:30.615 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-20T18:06:30.616 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-20T18:06:30.617 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 0 + 1 2026-02-20T18:06:30.619 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=1 2026-02-20T18:06:30.619 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-85808-0-256 td/test-erasure-eio 1 256 add 2026-02-20T18:06:30.619 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-85808-0-256 2026-02-20T18:06:30.619 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-20T18:06:30.619 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-20T18:06:30.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-20T18:06:30.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-20T18:06:30.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-20T18:06:30.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=256 2026-02-20T18:06:30.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-20T18:06:30.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=add 2026-02-20T18:06:30.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-20T18:06:30.620 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-85808-0-256 2026-02-20T18:06:30.620 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:06:30.620 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-85808-0-256 2026-02-20T18:06:30.620 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-85808-0-256 2026-02-20T18:06:30.621 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:06:30.847 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T18:06:30.847 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:06:30.848 INFO:tasks.workunit.client.0.trial195.stderr:2' 2026-02-20T18:06:30.848 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-20T18:06:30.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-20T18:06:30.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-20T18:06:30.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-20T18:06:30.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-20T18:06:31.165 INFO:tasks.workunit.client.0.trial195.stderr:noout is set 2026-02-20T18:06:31.175 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' add = add ']' 2026-02-20T18:06:31.175 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-85808-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:06:31.175 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-20T18:06:31.175 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-20T18:06:31.175 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-20T18:06:31.175 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-20T18:06:31.176 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-85808-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:06:31.176 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-20T18:06:31.176 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-20T18:06:31.176 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-20T18:06:31.176 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-20T18:06:31.176 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-20T18:06:31.176 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:06:31.176 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:06:31.176 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:06:31.176 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:06:31.176 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:06:31.482 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:06:31.482 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-85808-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:06:31.482 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-20T18:06:31.482 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-20T18:06:31.483 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-20T18:06:31.483 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-20T18:06:31.483 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-20T18:06:31.483 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-85808-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:06:31.564 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-20T18:06:31.564 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-20T18:06:31.565 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-20T18:06:31.565 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-20T18:06:31.565 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-20T18:06:31.565 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-20T18:06:31.565 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:06:31.565 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:06:31.565 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:06:31.565 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:06:31.565 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-20T18:06:31.565 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-20T18:06:31.566 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-20T18:06:31.566 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-20T18:06:31.566 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:06:31.566 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-20T18:06:31.566 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:06:31.566 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:06:31.566 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:06:31.566 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:06:31.566 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:06:31.566 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:06:31.566 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:06:31.566 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:06:31.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:06:31.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:06:31.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:06:31.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:06:31.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:06:31.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:06:31.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-20T18:06:31.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-20T18:06:31.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-20T18:06:31.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-20T18:06:31.567 INFO:tasks.workunit.client.0.trial195.stderr:start osd.1 2026-02-20T18:06:31.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:06:31.568 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-20T18:06:31.568 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-20T18:06:31.568 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-20T18:06:31.569 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-20T18:06:31.569 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-20T18:06:31.579 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:31.578+0000 7fa6b832b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:31.581 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:31.580+0000 7fa6b832b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:31.581 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:31.581+0000 7fa6b832b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:31.768 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:31.767+0000 7fa6b832b900 -1 Falling back to public interface 2026-02-20T18:06:31.797 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-20T18:06:31.797 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:06:31.797 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-20T18:06:31.797 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:06:31.797 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:06:31.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:06:31.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:06:31.798 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T18:06:31.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:06:31.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:06:32.027 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:06:32.098 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:32.097+0000 7fa6b832b900 -1 osd.1 87 log_to_monitors true 2026-02-20T18:06:33.028 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:06:33.029 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:06:33.029 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:06:33.029 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:06:33.029 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:06:33.029 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:06:33.253 INFO:tasks.workunit.client.0.trial195.stderr:osd.1 up in weight 1 up_from 92 up_thru 92 down_at 89 last_clean_interval [68,87) [v2:127.0.0.1:6810/953932935,v1:127.0.0.1:6811/953932935] [v2:127.0.0.1:6812/953932935,v1:127.0.0.1:6813/953932935] exists,up cf55ea6b-c46c-423c-b53a-234bec6c36cc 2026-02-20T18:06:33.253 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:06:33.253 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:06:33.253 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:06:33.254 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-20T18:06:33.254 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:06:33.254 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:06:33.254 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:06:33.254 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:06:33.255 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:06:33.255 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:06:33.255 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:06:33.255 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:06:33.255 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:06:33.342 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:06:33.342 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:06:33.342 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:06:33.342 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:06:33.342 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:06:33.343 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:06:33.566 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:06:33.566 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:06:33.567 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:06:33.567 INFO:tasks.workunit.client.0.trial195.stderr:3' 2026-02-20T18:06:33.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:06:33.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:06:33.567 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:06:33.632 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803804 2026-02-20T18:06:33.632 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803804 2026-02-20T18:06:33.632 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803804' 2026-02-20T18:06:33.632 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:06:33.632 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:06:33.695 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=395136991235 2026-02-20T18:06:33.695 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 395136991235 2026-02-20T18:06:33.695 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803804 1-395136991235' 2026-02-20T18:06:33.695 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:06:33.695 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:06:33.760 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444059 2026-02-20T18:06:33.760 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444059 2026-02-20T18:06:33.760 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803804 1-395136991235 2-73014444059' 2026-02-20T18:06:33.760 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:06:33.760 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:06:33.825 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=356482285574 2026-02-20T18:06:33.826 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 356482285574 2026-02-20T18:06:33.826 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803804 1-395136991235 2-73014444059 3-356482285574' 2026-02-20T18:06:33.826 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:06:33.826 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803804 2026-02-20T18:06:33.826 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:06:33.827 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:06:33.827 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803804 2026-02-20T18:06:33.828 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:06:33.829 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803804 2026-02-20T18:06:33.829 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803804' 2026-02-20T18:06:33.829 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.0 seq 25769803804 2026-02-20T18:06:33.830 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:06:34.052 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803801 -lt 25769803804 2026-02-20T18:06:34.052 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:06:35.053 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:06:35.053 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:06:35.285 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803804 -lt 25769803804 2026-02-20T18:06:35.285 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:06:35.286 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-395136991235 2026-02-20T18:06:35.286 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:06:35.286 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:06:35.287 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-395136991235 2026-02-20T18:06:35.287 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:06:35.288 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=395136991235 2026-02-20T18:06:35.288 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 395136991235' 2026-02-20T18:06:35.288 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.1 seq 395136991235 2026-02-20T18:06:35.288 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:06:35.519 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 395136991235 -lt 395136991235 2026-02-20T18:06:35.519 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:06:35.520 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444059 2026-02-20T18:06:35.520 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:06:35.520 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:06:35.521 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-73014444059 2026-02-20T18:06:35.521 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:06:35.522 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=73014444059 2026-02-20T18:06:35.522 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 73014444059' 2026-02-20T18:06:35.522 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.2 seq 73014444059 2026-02-20T18:06:35.523 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:06:35.751 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 73014444059 -lt 73014444059 2026-02-20T18:06:35.752 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:06:35.752 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-356482285574 2026-02-20T18:06:35.752 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:06:35.753 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:06:35.753 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-356482285574 2026-02-20T18:06:35.754 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:06:35.755 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=356482285574 2026-02-20T18:06:35.755 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 356482285574' 2026-02-20T18:06:35.755 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.3 seq 356482285574 2026-02-20T18:06:35.755 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:06:35.980 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 356482285574 -lt 356482285574 2026-02-20T18:06:35.981 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:06:35.981 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:06:35.981 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:06:36.292 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:06:36.292 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:06:36.292 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:06:36.292 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:06:36.292 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:06:36.292 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:06:36.293 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:06:36.293 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:06:36.513 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:06:36.514 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:06:36.514 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:06:36.514 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:06:36.826 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:06:36.826 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:06:36.826 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:06:36.826 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:219: set_size: dd if=/dev/urandom bs=256 count=1 2026-02-20T18:06:36.828 INFO:tasks.workunit.client.0.trial195.stderr:1+0 records in 2026-02-20T18:06:36.828 INFO:tasks.workunit.client.0.trial195.stderr:1+0 records out 2026-02-20T18:06:36.828 INFO:tasks.workunit.client.0.trial195.stderr:256 bytes copied, 2.0107e-05 s, 12.7 MB/s 2026-02-20T18:06:36.828 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-85808-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:06:36.828 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-20T18:06:36.828 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-20T18:06:36.829 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-20T18:06:36.829 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-20T18:06:36.829 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-85808-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:06:36.829 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-20T18:06:36.829 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-20T18:06:36.829 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-20T18:06:36.829 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-20T18:06:36.829 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-20T18:06:36.829 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:06:36.829 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:06:36.829 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:06:36.829 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:06:36.830 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:06:36.933 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:06:36.933 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-85808-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:06:36.934 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-20T18:06:36.934 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-20T18:06:36.934 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-20T18:06:36.934 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-20T18:06:36.934 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-20T18:06:36.934 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-85808-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:06:37.064 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-20T18:06:37.065 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-20T18:06:37.065 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-20T18:06:37.065 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-20T18:06:37.065 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-20T18:06:37.065 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-20T18:06:37.065 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:06:37.065 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:06:37.065 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:06:37.065 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:06:37.065 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-20T18:06:37.066 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-20T18:06:37.066 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-20T18:06:37.066 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-20T18:06:37.066 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:06:37.066 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-20T18:06:37.066 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:06:37.066 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:06:37.066 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:06:37.066 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:06:37.066 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:06:37.066 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:06:37.066 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:06:37.066 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:06:37.067 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:06:37.067 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:06:37.067 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:06:37.067 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:06:37.067 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:06:37.067 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:06:37.067 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-20T18:06:37.067 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-20T18:06:37.067 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-20T18:06:37.067 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-20T18:06:37.067 INFO:tasks.workunit.client.0.trial195.stderr:start osd.1 2026-02-20T18:06:37.068 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:06:37.068 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-20T18:06:37.069 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-20T18:06:37.069 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-20T18:06:37.069 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-20T18:06:37.069 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-20T18:06:37.080 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:37.079+0000 7f69f099f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:37.081 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:37.080+0000 7f69f099f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:37.082 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:37.081+0000 7f69f099f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:37.213 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:37.212+0000 7f69f099f900 -1 Falling back to public interface 2026-02-20T18:06:37.290 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-20T18:06:37.291 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:06:37.291 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-20T18:06:37.291 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:06:37.291 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:06:37.291 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:06:37.291 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:06:37.291 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T18:06:37.291 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:06:37.291 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:06:37.515 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:06:37.525 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:37.524+0000 7f69f099f900 -1 osd.1 93 log_to_monitors true 2026-02-20T18:06:38.517 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:06:38.517 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:06:38.517 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:06:38.517 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:06:38.517 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:06:38.518 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:06:38.743 INFO:tasks.workunit.client.0.trial195.stderr:osd.1 up in weight 1 up_from 97 up_thru 97 down_at 94 last_clean_interval [92,93) [v2:127.0.0.1:6810/3511889936,v1:127.0.0.1:6811/3511889936] [v2:127.0.0.1:6812/3511889936,v1:127.0.0.1:6813/3511889936] exists,up cf55ea6b-c46c-423c-b53a-234bec6c36cc 2026-02-20T18:06:38.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:06:38.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:06:38.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:06:38.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-20T18:06:38.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:06:38.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:06:38.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:06:38.744 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:06:38.745 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:06:38.745 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:06:38.746 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:06:38.746 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:06:38.746 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:06:38.830 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:06:38.830 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:06:38.831 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:06:38.831 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:06:38.831 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:06:38.831 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:06:39.057 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:06:39.057 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:06:39.057 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:06:39.057 INFO:tasks.workunit.client.0.trial195.stderr:3' 2026-02-20T18:06:39.057 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:06:39.058 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:06:39.058 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:06:39.123 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803807 2026-02-20T18:06:39.123 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803807 2026-02-20T18:06:39.123 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803807' 2026-02-20T18:06:39.123 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:06:39.123 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:06:39.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=416611827715 2026-02-20T18:06:39.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 416611827715 2026-02-20T18:06:39.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803807 1-416611827715' 2026-02-20T18:06:39.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:06:39.191 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:06:39.254 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444062 2026-02-20T18:06:39.254 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444062 2026-02-20T18:06:39.254 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803807 1-416611827715 2-73014444062' 2026-02-20T18:06:39.254 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:06:39.254 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:06:39.320 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=356482285577 2026-02-20T18:06:39.320 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 356482285577 2026-02-20T18:06:39.320 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803807 1-416611827715 2-73014444062 3-356482285577' 2026-02-20T18:06:39.320 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:06:39.320 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803807 2026-02-20T18:06:39.320 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:06:39.321 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:06:39.321 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803807 2026-02-20T18:06:39.321 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:06:39.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803807 2026-02-20T18:06:39.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803807' 2026-02-20T18:06:39.323 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.0 seq 25769803807 2026-02-20T18:06:39.323 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:06:39.547 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803805 -lt 25769803807 2026-02-20T18:06:39.547 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:06:40.548 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:06:40.549 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:06:40.779 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803807 -lt 25769803807 2026-02-20T18:06:40.779 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:06:40.780 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-416611827715 2026-02-20T18:06:40.780 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:06:40.781 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:06:40.781 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-416611827715 2026-02-20T18:06:40.781 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:06:40.783 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=416611827715 2026-02-20T18:06:40.783 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 416611827715' 2026-02-20T18:06:40.783 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.1 seq 416611827715 2026-02-20T18:06:40.783 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:06:41.006 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 416611827715 -lt 416611827715 2026-02-20T18:06:41.006 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:06:41.007 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444062 2026-02-20T18:06:41.007 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:06:41.008 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:06:41.008 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-73014444062 2026-02-20T18:06:41.008 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:06:41.009 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=73014444062 2026-02-20T18:06:41.009 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 73014444062' 2026-02-20T18:06:41.010 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.2 seq 73014444062 2026-02-20T18:06:41.010 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:06:41.233 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 73014444062 -lt 73014444062 2026-02-20T18:06:41.233 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:06:41.234 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-356482285577 2026-02-20T18:06:41.234 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:06:41.235 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:06:41.235 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-356482285577 2026-02-20T18:06:41.236 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:06:41.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=356482285577 2026-02-20T18:06:41.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 356482285577' 2026-02-20T18:06:41.237 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.3 seq 356482285577 2026-02-20T18:06:41.237 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:06:41.464 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 356482285577 -lt 356482285577 2026-02-20T18:06:41.465 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:06:41.465 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:06:41.465 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:06:41.778 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:06:41.779 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:06:41.779 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:06:41.779 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:06:41.779 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:06:41.779 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:06:41.779 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:06:41.779 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:06:42.000 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:06:42.001 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:06:42.001 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:06:42.001 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:06:42.315 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:06:42.315 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:06:42.315 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:06:42.315 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-20T18:06:42.316 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-20T18:06:42.667 INFO:tasks.workunit.client.0.trial195.stderr:noout is unset 2026-02-20T18:06:42.678 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 0 -eq 1 -a 1 -eq 2 ']' 2026-02-20T18:06:42.678 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:262: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-85808-0-256 fail 2026-02-20T18:06:42.678 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-20T18:06:42.678 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-20T18:06:42.678 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-85808-0-256 2026-02-20T18:06:42.678 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-20T18:06:42.678 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-20T18:06:42.678 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-size-85808-0-256 td/test-erasure-eio/COPY 2026-02-20T18:06:42.698 INFO:tasks.workunit.client.0.trial195.stderr:error getting pool-jerasure/obj-size-85808-0-256: (5) Input/output error 2026-02-20T18:06:42.700 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-20T18:06:42.700 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-20T18:06:42.701 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:332: TEST_rados_get_bad_size_shard_0: delete_erasure_coded_pool pool-jerasure 2026-02-20T18:06:42.701 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-20T18:06:42.701 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-20T18:06:42.930 INFO:tasks.workunit.client.0.trial195.stderr:pool 'pool-jerasure' does not exist 2026-02-20T18:06:42.937 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-20T18:06:43.212 INFO:tasks.workunit.client.0.trial195.stderr:erasure-code-profile myprofile does not exist 2026-02-20T18:06:43.221 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-20T18:06:43.221 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-20T18:06:43.221 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T18:06:43.221 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-20T18:06:43.221 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:06:43.222 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:06:43.222 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:06:43.222 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:06:43.222 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:06:43.330 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:06:43.331 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T18:06:43.332 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T18:06:43.332 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T18:06:43.334 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T18:06:43.334 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T18:06:43.334 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T18:06:43.335 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:06:43.335 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T18:06:43.336 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T18:06:43.336 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:06:43.337 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T18:06:43.338 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T18:06:43.338 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-20T18:06:43.354 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T18:06:43.354 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:06:43.355 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:06:43.355 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.85808 2026-02-20T18:06:43.356 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T18:06:43.357 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T18:06:43.357 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-20T18:06:43.357 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-20T18:06:43.357 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-20T18:06:43.357 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-20T18:06:43.357 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-20T18:06:43.357 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T18:06:43.357 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-20T18:06:43.357 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:06:43.357 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:06:43.357 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:06:43.358 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:06:43.358 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:06:43.358 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:06:43.358 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T18:06:43.359 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T18:06:43.359 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T18:06:43.360 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T18:06:43.360 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T18:06:43.361 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T18:06:43.362 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:06:43.362 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T18:06:43.362 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T18:06:43.363 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:06:43.363 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T18:06:43.364 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T18:06:43.364 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-20T18:06:43.365 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T18:06:43.365 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:06:43.365 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:06:43.365 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.85808 2026-02-20T18:06:43.366 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T18:06:43.366 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T18:06:43.366 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-20T18:06:43.367 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-20T18:06:43.367 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:06:43.368 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:06:43.368 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.85808 2026-02-20T18:06:43.369 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-20T18:06:43.370 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-20T18:06:43.370 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-20T18:06:43.370 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-20T18:06:43.370 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-20T18:06:43.370 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-20T18:06:43.370 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-20T18:06:43.370 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-20T18:06:43.371 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-20T18:06:43.371 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-20T18:06:43.371 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-20T18:06:43.391 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-20T18:06:43.391 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:06:43.391 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:06:43.392 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:06:43.392 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:06:43.392 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:06:43.392 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:06:43.392 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-20T18:06:43.419 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-20T18:06:43.421 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-20T18:06:43.421 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T18:06:43.421 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T18:06:43.422 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-20T18:06:43.422 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-20T18:06:43.422 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T18:06:43.422 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:06:43.422 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:06:43.422 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:06:43.422 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:06:43.422 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:06:43.422 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:06:43.423 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T18:06:43.423 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.85808/ceph-mon.a.asok config get fsid 2026-02-20T18:06:43.467 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-20T18:06:43.467 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T18:06:43.468 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T18:06:43.468 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-20T18:06:43.468 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-20T18:06:43.468 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T18:06:43.468 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:06:43.468 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:06:43.469 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:06:43.469 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:06:43.469 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:06:43.469 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:06:43.469 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T18:06:43.470 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.85808/ceph-mon.a.asok config get mon_host 2026-02-20T18:06:43.513 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-20T18:06:43.513 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-20T18:06:43.513 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-20T18:06:43.513 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-20T18:06:43.513 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-20T18:06:43.513 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-20T18:06:43.513 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-20T18:06:43.615 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-20T18:06:43.616 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:06:43.616 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:06:43.616 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:06:43.616 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:06:43.616 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:06:43.616 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:06:43.617 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T18:06:43.617 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T18:06:43.635 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-20T18:06:43.635 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-20T18:06:43.732 INFO:tasks.workunit.client.0.trial195.stderr:pool 'rbd' already exists 2026-02-20T18:06:43.739 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-20T18:06:44.741 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-20T18:06:44.742 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:06:44.742 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:06:44.742 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:06:44.742 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:06:44.742 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:06:44.742 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:06:44.742 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-20T18:06:44.742 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-mon.a.asok log flush 2026-02-20T18:06:44.783 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T18:06:44.788 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-20T18:06:44.789 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T18:06:43.410+0000 7fcac5dc8d80 0 load: jerasure load: lrc 2026-02-20T18:06:44.790 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_bad_size_shard_1 td/test-erasure-eio 2026-02-20T18:06:44.790 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:336: TEST_rados_get_bad_size_shard_1: local dir=td/test-erasure-eio 2026-02-20T18:06:44.790 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:337: TEST_rados_get_bad_size_shard_1: setup_osds 4 2026-02-20T18:06:44.790 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-20T18:06:44.790 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-20T18:06:44.790 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-20T18:06:44.791 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-20T18:06:44.794 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:06:44.795 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-20T18:06:44.795 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:06:44.795 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:06:44.795 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-20T18:06:44.795 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:06:44.795 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-20T18:06:44.795 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:06:44.795 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:06:44.796 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:06:44.796 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:06:44.796 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-20T18:06:44.796 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-20T18:06:44.796 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:06:44.796 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:06:44.796 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:06:44.796 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:06:44.796 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:06:44.796 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:06:44.796 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:06:44.797 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:06:44.797 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:06:44.797 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:06:44.797 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:06:44.797 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:06:44.797 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:06:44.797 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:06:44.797 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:06:44.797 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:06:44.797 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:06:44.797 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:06:44.797 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:06:44.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:06:44.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:06:44.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:06:44.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-20T18:06:44.798 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:06:44.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ab9bad50-85b6-4f62-9db6-0086ed6bacc2 2026-02-20T18:06:44.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 ab9bad50-85b6-4f62-9db6-0086ed6bacc2' 2026-02-20T18:06:44.798 INFO:tasks.workunit.client.0.trial195.stdout:add osd0 ab9bad50-85b6-4f62-9db6-0086ed6bacc2 2026-02-20T18:06:44.799 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:06:44.807 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC0ophp9hQHMBAACHtsdm7XmLmN8hkFvgrj2A== 2026-02-20T18:06:44.807 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC0ophp9hQHMBAACHtsdm7XmLmN8hkFvgrj2A=="}' 2026-02-20T18:06:44.807 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ab9bad50-85b6-4f62-9db6-0086ed6bacc2 -i td/test-erasure-eio/0/new.json 2026-02-20T18:06:44.911 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:06:44.918 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-20T18:06:44.919 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC0ophp9hQHMBAACHtsdm7XmLmN8hkFvgrj2A== --osd-uuid ab9bad50-85b6-4f62-9db6-0086ed6bacc2 2026-02-20T18:06:44.938 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:44.936+0000 7f01bda25900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:44.939 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:44.937+0000 7f01bda25900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:44.940 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:44.938+0000 7f01bda25900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:44.941 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:44.938+0000 7f01bda25900 -1 bdev(0x55fab4114c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-20T18:06:44.941 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:44.939+0000 7f01bda25900 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-20T18:06:45.344 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-20T18:06:45.345 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:06:45.345 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-20T18:06:45.346 INFO:tasks.workunit.client.0.trial195.stdout:adding osd0 key to auth repository 2026-02-20T18:06:45.346 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:06:45.454 INFO:tasks.workunit.client.0.trial195.stdout:start osd.0 2026-02-20T18:06:45.454 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-20T18:06:45.455 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:06:45.455 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:06:45.455 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:06:45.455 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:06:45.473 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:45.471+0000 7f27e5b9f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:45.474 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:45.472+0000 7f27e5b9f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:45.475 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:45.473+0000 7f27e5b9f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:45.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-20T18:06:45.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:06:45.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-20T18:06:45.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:06:45.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:06:45.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:06:45.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:06:45.555 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:06:45.555 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:06:45.555 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T18:06:45.655 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:06:45.679 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:45.677+0000 7f27e5b9f900 -1 Falling back to public interface 2026-02-20T18:06:45.959 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:45.957+0000 7f27e5b9f900 -1 osd.0 0 log_to_monitors true 2026-02-20T18:06:46.656 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:06:46.657 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:06:46.657 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:06:46.658 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:06:46.658 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:06:46.658 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T18:06:46.883 INFO:tasks.workunit.client.0.trial195.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1228379666,v1:127.0.0.1:6803/1228379666] [v2:127.0.0.1:6804/1228379666,v1:127.0.0.1:6805/1228379666] exists,up ab9bad50-85b6-4f62-9db6-0086ed6bacc2 2026-02-20T18:06:46.887 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:06:46.887 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:06:46.887 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:06:46.887 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:06:46.887 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-20T18:06:46.887 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:06:46.887 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:06:46.887 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-20T18:06:46.887 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:06:46.887 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-20T18:06:46.888 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:06:46.888 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:06:46.888 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:06:46.888 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:06:46.888 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-20T18:06:46.888 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-20T18:06:46.888 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:06:46.888 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:06:46.888 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:06:46.888 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:06:46.888 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:06:46.889 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:06:46.889 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:06:46.889 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:06:46.889 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:06:46.889 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:06:46.889 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:06:46.889 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:06:46.889 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:06:46.889 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:06:46.889 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:06:46.889 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:06:46.889 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:06:46.889 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:06:46.889 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:06:46.890 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:06:46.890 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:06:46.890 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:06:46.890 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-20T18:06:46.890 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:06:46.890 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=56e4a9c1-3b40-41fc-b068-854b88a18a1b 2026-02-20T18:06:46.891 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 56e4a9c1-3b40-41fc-b068-854b88a18a1b' 2026-02-20T18:06:46.891 INFO:tasks.workunit.client.0.trial195.stdout:add osd1 56e4a9c1-3b40-41fc-b068-854b88a18a1b 2026-02-20T18:06:46.891 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:06:46.901 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC2ophpX8WZNRAAdW9qYaZdg6WrEaQAq8fr9Q== 2026-02-20T18:06:46.901 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC2ophpX8WZNRAAdW9qYaZdg6WrEaQAq8fr9Q=="}' 2026-02-20T18:06:46.901 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 56e4a9c1-3b40-41fc-b068-854b88a18a1b -i td/test-erasure-eio/1/new.json 2026-02-20T18:06:47.132 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:06:47.140 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-20T18:06:47.141 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC2ophpX8WZNRAAdW9qYaZdg6WrEaQAq8fr9Q== --osd-uuid 56e4a9c1-3b40-41fc-b068-854b88a18a1b 2026-02-20T18:06:47.158 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:47.156+0000 7f0cbd305900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:47.159 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:47.158+0000 7f0cbd305900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:47.161 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:47.159+0000 7f0cbd305900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:47.161 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:47.159+0000 7f0cbd305900 -1 bdev(0x55eccc1d8c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-20T18:06:47.161 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:47.159+0000 7f0cbd305900 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-20T18:06:47.490 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-20T18:06:47.490 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:06:47.490 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-20T18:06:47.491 INFO:tasks.workunit.client.0.trial195.stdout:adding osd1 key to auth repository 2026-02-20T18:06:47.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:06:47.807 INFO:tasks.workunit.client.0.trial195.stdout:start osd.1 2026-02-20T18:06:47.807 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-20T18:06:47.807 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:06:47.808 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:06:47.808 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:06:47.808 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:06:47.824 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:47.822+0000 7f3225022900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:47.825 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:47.824+0000 7f3225022900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:47.826 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:47.824+0000 7f3225022900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:47.969 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:47.966+0000 7f3225022900 -1 Falling back to public interface 2026-02-20T18:06:48.031 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-20T18:06:48.031 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:06:48.031 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-20T18:06:48.031 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:06:48.031 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:06:48.032 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:06:48.032 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:06:48.032 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:06:48.032 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:06:48.032 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:06:48.259 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:06:48.316 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:48.315+0000 7f3225022900 -1 osd.1 0 log_to_monitors true 2026-02-20T18:06:49.261 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:06:49.261 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:06:49.262 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:06:49.262 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:06:49.262 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:06:49.263 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:06:49.484 INFO:tasks.workunit.client.0.trial195.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1070023783,v1:127.0.0.1:6811/1070023783] [v2:127.0.0.1:6812/1070023783,v1:127.0.0.1:6813/1070023783] exists,up 56e4a9c1-3b40-41fc-b068-854b88a18a1b 2026-02-20T18:06:49.488 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:06:49.488 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:06:49.488 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:06:49.488 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:06:49.488 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-20T18:06:49.488 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:06:49.489 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:06:49.489 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-20T18:06:49.489 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:06:49.489 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-20T18:06:49.489 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:06:49.489 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:06:49.489 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:06:49.489 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:06:49.489 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-20T18:06:49.489 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-20T18:06:49.489 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:06:49.489 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:06:49.490 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:06:49.490 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:06:49.490 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:06:49.490 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:06:49.490 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:06:49.490 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:06:49.490 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:06:49.490 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:06:49.490 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:06:49.490 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:06:49.490 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:06:49.490 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:06:49.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:06:49.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:06:49.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:06:49.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:06:49.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:06:49.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:06:49.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:06:49.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:06:49.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-20T18:06:49.491 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:06:49.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=3c193710-aac4-4283-800c-b68162ff83c5 2026-02-20T18:06:49.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 3c193710-aac4-4283-800c-b68162ff83c5' 2026-02-20T18:06:49.492 INFO:tasks.workunit.client.0.trial195.stdout:add osd2 3c193710-aac4-4283-800c-b68162ff83c5 2026-02-20T18:06:49.492 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:06:49.500 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC5ophpF1a/HRAAJOMwniJcxSg5eL9I36BLqQ== 2026-02-20T18:06:49.500 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC5ophpF1a/HRAAJOMwniJcxSg5eL9I36BLqQ=="}' 2026-02-20T18:06:49.500 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 3c193710-aac4-4283-800c-b68162ff83c5 -i td/test-erasure-eio/2/new.json 2026-02-20T18:06:49.725 INFO:tasks.workunit.client.0.trial195.stdout:2 2026-02-20T18:06:49.732 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-20T18:06:49.733 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC5ophpF1a/HRAAJOMwniJcxSg5eL9I36BLqQ== --osd-uuid 3c193710-aac4-4283-800c-b68162ff83c5 2026-02-20T18:06:49.751 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:49.749+0000 7fbb53663900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:49.753 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:49.751+0000 7fbb53663900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:49.753 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:49.752+0000 7fbb53663900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:49.754 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:49.752+0000 7fbb53663900 -1 bdev(0x55872f5f8c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-20T18:06:49.754 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:49.752+0000 7fbb53663900 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-20T18:06:50.077 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-20T18:06:50.078 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:06:50.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-20T18:06:50.079 INFO:tasks.workunit.client.0.trial195.stdout:adding osd2 key to auth repository 2026-02-20T18:06:50.080 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:06:50.394 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-20T18:06:50.395 INFO:tasks.workunit.client.0.trial195.stdout:start osd.2 2026-02-20T18:06:50.396 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:06:50.396 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:06:50.396 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:06:50.396 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:06:50.411 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:50.409+0000 7f322bc67900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:50.412 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:50.410+0000 7f322bc67900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:50.413 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:50.411+0000 7f322bc67900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:50.544 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:50.542+0000 7f322bc67900 -1 Falling back to public interface 2026-02-20T18:06:50.619 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-20T18:06:50.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:06:50.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-20T18:06:50.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:06:50.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:06:50.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:06:50.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:06:50.620 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:06:50.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:06:50.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:06:50.846 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:06:50.883 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:50.881+0000 7f322bc67900 -1 osd.2 0 log_to_monitors true 2026-02-20T18:06:51.847 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:06:51.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:06:51.848 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:06:51.849 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:06:51.849 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:06:51.849 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:06:52.072 INFO:tasks.workunit.client.0.trial195.stdout:osd.2 up in weight 1 up_from 18 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/474962983,v1:127.0.0.1:6819/474962983] [v2:127.0.0.1:6820/474962983,v1:127.0.0.1:6821/474962983] exists,up 3c193710-aac4-4283-800c-b68162ff83c5 2026-02-20T18:06:52.076 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:06:52.076 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:06:52.076 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:06:52.076 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:06:52.077 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-20T18:06:52.077 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:06:52.077 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:06:52.077 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-20T18:06:52.077 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:06:52.077 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-20T18:06:52.077 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:06:52.077 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:06:52.077 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:06:52.077 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:06:52.077 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-20T18:06:52.077 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-20T18:06:52.078 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:06:52.078 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:06:52.078 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:06:52.078 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:06:52.078 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:06:52.078 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:06:52.078 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:06:52.078 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:06:52.078 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:06:52.078 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:06:52.078 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:06:52.078 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:06:52.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:06:52.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:06:52.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:06:52.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:06:52.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:06:52.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:06:52.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:06:52.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:06:52.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:06:52.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:06:52.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-20T18:06:52.079 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:06:52.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8d07e028-6957-428c-be9a-530717a6ff36 2026-02-20T18:06:52.080 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 8d07e028-6957-428c-be9a-530717a6ff36' 2026-02-20T18:06:52.080 INFO:tasks.workunit.client.0.trial195.stdout:add osd3 8d07e028-6957-428c-be9a-530717a6ff36 2026-02-20T18:06:52.080 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:06:52.090 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC8ophp1W5HBRAAXJhe1byH7+VGrH6WGN+PwQ== 2026-02-20T18:06:52.090 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC8ophp1W5HBRAAXJhe1byH7+VGrH6WGN+PwQ=="}' 2026-02-20T18:06:52.090 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8d07e028-6957-428c-be9a-530717a6ff36 -i td/test-erasure-eio/3/new.json 2026-02-20T18:06:52.318 INFO:tasks.workunit.client.0.trial195.stdout:3 2026-02-20T18:06:52.325 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-20T18:06:52.326 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC8ophp1W5HBRAAXJhe1byH7+VGrH6WGN+PwQ== --osd-uuid 8d07e028-6957-428c-be9a-530717a6ff36 2026-02-20T18:06:52.343 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:52.341+0000 7f1fb080d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:52.345 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:52.343+0000 7f1fb080d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:52.345 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:52.344+0000 7f1fb080d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:52.346 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:52.344+0000 7f1fb080d900 -1 bdev(0x55f48b6eec00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-20T18:06:52.346 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:52.344+0000 7f1fb080d900 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-20T18:06:52.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-20T18:06:52.671 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:06:52.671 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-20T18:06:52.671 INFO:tasks.workunit.client.0.trial195.stdout:adding osd3 key to auth repository 2026-02-20T18:06:52.672 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:06:52.990 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-20T18:06:52.991 INFO:tasks.workunit.client.0.trial195.stdout:start osd.3 2026-02-20T18:06:52.992 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:06:52.992 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:06:52.992 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:06:52.993 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:06:53.007 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:53.005+0000 7f0648c10900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:53.008 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:53.006+0000 7f0648c10900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:53.009 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:53.007+0000 7f0648c10900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:06:53.186 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:53.184+0000 7f0648c10900 -1 Falling back to public interface 2026-02-20T18:06:53.218 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-20T18:06:53.218 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:06:53.218 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-20T18:06:53.218 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:06:53.218 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:06:53.218 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:06:53.218 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:06:53.219 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:06:53.219 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:06:53.219 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T18:06:53.442 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:06:53.558 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:06:53.556+0000 7f0648c10900 -1 osd.3 0 log_to_monitors true 2026-02-20T18:06:54.443 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:06:54.443 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:06:54.444 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:06:54.444 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:06:54.444 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:06:54.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T18:06:54.668 INFO:tasks.workunit.client.0.trial195.stdout:osd.3 up in weight 1 up_from 26 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/286180415,v1:127.0.0.1:6827/286180415] [v2:127.0.0.1:6828/286180415,v1:127.0.0.1:6829/286180415] exists,up 8d07e028-6957-428c-be9a-530717a6ff36 2026-02-20T18:06:54.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:06:54.671 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:06:54.671 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:06:54.671 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-20T18:06:54.671 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-20T18:06:54.671 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-20T18:06:54.671 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:06:54.672 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:06:54.672 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:06:54.672 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.0.asok 2026-02-20T18:06:54.672 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-20T18:06:54.672 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-osd.0.asok log flush 2026-02-20T18:06:54.714 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T18:06:54.717 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-20T18:06:54.719 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T18:06:45.680+0000 7f27e5b9f900 0 load: jerasure load: lrc 2026-02-20T18:06:54.721 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:339: TEST_rados_get_bad_size_shard_1: local poolname=pool-jerasure 2026-02-20T18:06:54.721 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:340: TEST_rados_get_bad_size_shard_1: create_erasure_coded_pool pool-jerasure 2 1 2026-02-20T18:06:54.721 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-20T18:06:54.721 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-20T18:06:54.722 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-20T18:06:54.722 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-20T18:06:54.722 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-20T18:06:54.722 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-20T18:06:54.722 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-20T18:06:55.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-20T18:06:55.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-20T18:06:55.396 INFO:tasks.workunit.client.0.trial195.stderr:pool 'pool-jerasure' already exists 2026-02-20T18:06:55.404 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-20T18:06:56.406 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-20T18:06:56.406 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:06:56.406 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:06:56.406 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:06:56.406 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:06:56.407 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:06:56.407 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:06:56.407 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:06:56.408 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:06:56.408 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:06:56.496 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:06:56.497 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:06:56.497 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:06:56.497 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:06:56.497 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:06:56.497 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:06:56.723 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:06:56.723 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:06:56.724 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:06:56.724 INFO:tasks.workunit.client.0.trial195.stderr:3' 2026-02-20T18:06:56.724 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:06:56.724 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:06:56.724 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:06:56.789 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803780 2026-02-20T18:06:56.789 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803780 2026-02-20T18:06:56.789 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803780' 2026-02-20T18:06:56.789 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:06:56.789 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:06:56.855 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607557 2026-02-20T18:06:56.855 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607557 2026-02-20T18:06:56.855 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803780 1-51539607557' 2026-02-20T18:06:56.856 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:06:56.856 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:06:56.918 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411331 2026-02-20T18:06:56.918 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411331 2026-02-20T18:06:56.918 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803780 1-51539607557 2-77309411331' 2026-02-20T18:06:56.918 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:06:56.919 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:06:56.984 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149699 2026-02-20T18:06:56.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149699 2026-02-20T18:06:56.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803780 1-51539607557 2-77309411331 3-111669149699' 2026-02-20T18:06:56.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:06:56.985 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803780 2026-02-20T18:06:56.985 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:06:56.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:06:56.986 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803780 2026-02-20T18:06:56.986 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:06:56.987 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803780 2026-02-20T18:06:56.987 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803780' 2026-02-20T18:06:56.988 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.0 seq 25769803780 2026-02-20T18:06:56.988 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:06:57.211 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803778 -lt 25769803780 2026-02-20T18:06:57.211 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:06:58.212 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:06:58.213 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:06:58.439 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803781 -lt 25769803780 2026-02-20T18:06:58.439 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:06:58.439 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607557 2026-02-20T18:06:58.439 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:06:58.440 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:06:58.441 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-51539607557 2026-02-20T18:06:58.441 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:06:58.442 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=51539607557 2026-02-20T18:06:58.442 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 51539607557' 2026-02-20T18:06:58.442 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.1 seq 51539607557 2026-02-20T18:06:58.443 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:06:58.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 51539607557 -lt 51539607557 2026-02-20T18:06:58.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:06:58.670 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411331 2026-02-20T18:06:58.671 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:06:58.671 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:06:58.672 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-77309411331 2026-02-20T18:06:58.672 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:06:58.673 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=77309411331 2026-02-20T18:06:58.673 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 77309411331' 2026-02-20T18:06:58.674 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.2 seq 77309411331 2026-02-20T18:06:58.674 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:06:58.899 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411332 -lt 77309411331 2026-02-20T18:06:58.899 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:06:58.899 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149699 2026-02-20T18:06:58.900 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:06:58.900 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:06:58.901 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149699 2026-02-20T18:06:58.901 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:06:58.902 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149699 2026-02-20T18:06:58.902 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149699' 2026-02-20T18:06:58.902 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.3 seq 111669149699 2026-02-20T18:06:58.902 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:06:59.125 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149699 -lt 111669149699 2026-02-20T18:06:59.125 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:06:59.125 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:06:59.125 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:06:59.437 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:06:59.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:06:59.438 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:06:59.438 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:06:59.438 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:06:59.438 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:06:59.438 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:06:59.438 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:06:59.663 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:06:59.664 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:06:59.665 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:06:59.665 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:06:59.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:06:59.986 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:06:59.986 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:06:59.986 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:342: TEST_rados_get_bad_size_shard_1: local shard_id=1 2026-02-20T18:06:59.986 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:343: TEST_rados_get_bad_size_shard_1: rados_get_data_bad_size td/test-erasure-eio 1 10 2026-02-20T18:06:59.986 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-20T18:06:59.986 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-20T18:06:59.987 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=1 2026-02-20T18:06:59.987 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-20T18:06:59.987 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=10 2026-02-20T18:06:59.987 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-20T18:06:59.987 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=set 2026-02-20T18:06:59.987 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-20T18:06:59.987 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-85808-1-10 2026-02-20T18:06:59.987 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-85808-1-10 2026-02-20T18:06:59.987 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-20T18:06:59.988 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-20T18:06:59.988 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-85808-1-10 2026-02-20T18:06:59.988 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:06:59.988 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-20T18:06:59.988 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:06:59.988 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-20T18:06:59.988 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:06:59.988 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-20T18:06:59.988 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:06:59.989 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-20T18:06:59.989 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-85808-1-10 td/test-erasure-eio/ORIGINAL 2026-02-20T18:07:00.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-85808-1-10 td/test-erasure-eio 1 10 set 2026-02-20T18:07:00.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-85808-1-10 2026-02-20T18:07:00.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-20T18:07:00.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-20T18:07:00.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-20T18:07:00.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-20T18:07:00.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-20T18:07:00.005 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=10 2026-02-20T18:07:00.005 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-20T18:07:00.005 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-20T18:07:00.005 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-20T18:07:00.005 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-85808-1-10 2026-02-20T18:07:00.005 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:07:00.005 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-85808-1-10 2026-02-20T18:07:00.005 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-85808-1-10 2026-02-20T18:07:00.005 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:07:00.231 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T18:07:00.231 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:07:00.231 INFO:tasks.workunit.client.0.trial195.stderr:2' 2026-02-20T18:07:00.232 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-20T18:07:00.232 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-20T18:07:00.232 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-20T18:07:00.232 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-20T18:07:00.232 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-20T18:07:00.515 INFO:tasks.workunit.client.0.trial195.stderr:noout is set 2026-02-20T18:07:00.525 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-20T18:07:00.525 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 10 = 0 ']' 2026-02-20T18:07:00.525 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:224: set_size: dd if=/dev/urandom bs=10 count=1 of=td/test-erasure-eio/CORRUPT 2026-02-20T18:07:00.527 INFO:tasks.workunit.client.0.trial195.stderr:1+0 records in 2026-02-20T18:07:00.527 INFO:tasks.workunit.client.0.trial195.stderr:1+0 records out 2026-02-20T18:07:00.527 INFO:tasks.workunit.client.0.trial195.stderr:10 bytes copied, 3.7169e-05 s, 269 kB/s 2026-02-20T18:07:00.527 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-85808-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:07:00.527 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-20T18:07:00.527 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-20T18:07:00.527 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-20T18:07:00.528 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-20T18:07:00.528 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-85808-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:07:00.528 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-20T18:07:00.528 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-20T18:07:00.528 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-20T18:07:00.528 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-20T18:07:00.528 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-20T18:07:00.528 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:07:00.528 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:07:00.528 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:07:00.528 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:07:00.528 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:07:00.835 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:07:00.835 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-85808-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:07:00.835 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-20T18:07:00.835 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-20T18:07:00.835 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-20T18:07:00.835 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-20T18:07:00.835 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-20T18:07:00.835 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-85808-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:07:00.973 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-20T18:07:00.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-20T18:07:00.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-20T18:07:00.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-20T18:07:00.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-20T18:07:00.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-20T18:07:00.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:07:00.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:07:00.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:07:00.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:07:00.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-20T18:07:00.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-20T18:07:00.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-20T18:07:00.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-20T18:07:00.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:07:00.974 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-20T18:07:00.974 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:07:00.974 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:07:00.974 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:07:00.974 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:07:00.974 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:07:00.974 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:07:00.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:07:00.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:07:00.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:07:00.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:07:00.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:07:00.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:07:00.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:07:00.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:07:00.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-20T18:07:00.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-20T18:07:00.975 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-20T18:07:00.976 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-20T18:07:00.976 INFO:tasks.workunit.client.0.trial195.stderr:start osd.1 2026-02-20T18:07:00.976 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:07:00.976 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-20T18:07:00.977 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-20T18:07:00.977 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-20T18:07:00.977 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-20T18:07:00.977 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-20T18:07:00.992 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:00.990+0000 7f13ac81b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:00.993 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:00.992+0000 7f13ac81b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:00.994 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:00.993+0000 7f13ac81b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:01.129 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:01.127+0000 7f13ac81b900 -1 Falling back to public interface 2026-02-20T18:07:01.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-20T18:07:01.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:07:01.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-20T18:07:01.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:07:01.205 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:07:01.205 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:07:01.205 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:07:01.205 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T18:07:01.205 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:07:01.205 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:07:01.430 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:07:01.490 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:01.488+0000 7f13ac81b900 -1 osd.1 33 log_to_monitors true 2026-02-20T18:07:02.432 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:07:02.432 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:07:02.432 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:07:02.432 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:07:02.432 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:07:02.433 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:07:02.658 INFO:tasks.workunit.client.0.trial195.stderr:osd.1 up in weight 1 up_from 39 up_thru 39 down_at 36 last_clean_interval [12,33) [v2:127.0.0.1:6810/3353115615,v1:127.0.0.1:6811/3353115615] [v2:127.0.0.1:6812/3353115615,v1:127.0.0.1:6813/3353115615] exists,up 56e4a9c1-3b40-41fc-b068-854b88a18a1b 2026-02-20T18:07:02.658 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:07:02.658 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:07:02.658 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:07:02.658 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-20T18:07:02.658 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:07:02.658 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:07:02.658 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:07:02.658 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:07:02.659 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:07:02.659 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:07:02.659 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:07:02.659 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:07:02.659 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:07:02.743 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:07:02.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:07:02.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:07:02.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:07:02.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:07:02.744 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:07:02.970 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:07:02.970 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:07:02.971 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:07:02.971 INFO:tasks.workunit.client.0.trial195.stderr:3' 2026-02-20T18:07:02.971 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:07:02.971 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:07:02.971 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:07:03.038 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803784 2026-02-20T18:07:03.038 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803784 2026-02-20T18:07:03.038 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803784' 2026-02-20T18:07:03.039 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:07:03.039 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:07:03.101 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724547 2026-02-20T18:07:03.101 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724547 2026-02-20T18:07:03.101 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803784 1-167503724547' 2026-02-20T18:07:03.101 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:07:03.102 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:07:03.166 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411335 2026-02-20T18:07:03.166 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411335 2026-02-20T18:07:03.166 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803784 1-167503724547 2-77309411335' 2026-02-20T18:07:03.166 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:07:03.166 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:07:03.230 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149702 2026-02-20T18:07:03.231 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149702 2026-02-20T18:07:03.231 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803784 1-167503724547 2-77309411335 3-111669149702' 2026-02-20T18:07:03.231 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:07:03.231 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803784 2026-02-20T18:07:03.231 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:07:03.231 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:07:03.232 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803784 2026-02-20T18:07:03.232 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:07:03.233 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803784 2026-02-20T18:07:03.233 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-20T18:07:03.233 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.0 seq 25769803784 2026-02-20T18:07:03.234 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:07:03.458 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803781 -lt 25769803784 2026-02-20T18:07:03.458 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:07:04.458 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:07:04.459 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:07:04.686 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803784 -lt 25769803784 2026-02-20T18:07:04.686 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:07:04.687 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-167503724547 2026-02-20T18:07:04.687 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:07:04.688 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:07:04.688 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-167503724547 2026-02-20T18:07:04.688 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:07:04.689 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=167503724547 2026-02-20T18:07:04.690 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 167503724547' 2026-02-20T18:07:04.690 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.1 seq 167503724547 2026-02-20T18:07:04.690 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:07:04.919 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 167503724547 -lt 167503724547 2026-02-20T18:07:04.919 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:07:04.920 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411335 2026-02-20T18:07:04.920 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:07:04.920 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:07:04.921 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-77309411335 2026-02-20T18:07:04.921 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:07:04.922 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=77309411335 2026-02-20T18:07:04.922 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 77309411335' 2026-02-20T18:07:04.922 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.2 seq 77309411335 2026-02-20T18:07:04.923 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:07:05.144 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411335 -lt 77309411335 2026-02-20T18:07:05.144 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:07:05.145 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149702 2026-02-20T18:07:05.145 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:07:05.146 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:07:05.146 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149702 2026-02-20T18:07:05.147 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:07:05.148 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149702 2026-02-20T18:07:05.148 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149702' 2026-02-20T18:07:05.148 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.3 seq 111669149702 2026-02-20T18:07:05.148 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:07:05.374 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149702 -lt 111669149702 2026-02-20T18:07:05.375 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:07:05.375 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:07:05.376 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:07:05.692 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:07:05.692 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:07:05.693 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:07:05.693 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:07:05.693 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:07:05.693 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:07:05.693 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:07:05.693 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:07:05.924 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:07:05.924 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:07:05.925 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:07:05.925 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:07:06.240 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:07:06.240 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:07:06.240 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:07:06.240 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-20T18:07:06.241 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-20T18:07:06.560 INFO:tasks.workunit.client.0.trial195.stderr:noout is unset 2026-02-20T18:07:06.569 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-85808-1-10 2026-02-20T18:07:06.569 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-20T18:07:06.570 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-20T18:07:06.570 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-85808-1-10 2026-02-20T18:07:06.570 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-20T18:07:06.570 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-20T18:07:06.570 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-85808-1-10 td/test-erasure-eio/COPY 2026-02-20T18:07:06.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-20T18:07:06.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-20T18:07:06.592 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 1 + 1 2026-02-20T18:07:06.594 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=2 2026-02-20T18:07:06.594 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-85808-1-10 td/test-erasure-eio 2 10 set 2026-02-20T18:07:06.594 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-85808-1-10 2026-02-20T18:07:06.595 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-20T18:07:06.595 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-20T18:07:06.595 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-20T18:07:06.595 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=2 2026-02-20T18:07:06.595 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-20T18:07:06.595 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=10 2026-02-20T18:07:06.595 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-20T18:07:06.595 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-20T18:07:06.595 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-20T18:07:06.595 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-85808-1-10 2026-02-20T18:07:06.595 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:07:06.596 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-85808-1-10 2026-02-20T18:07:06.596 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-85808-1-10 2026-02-20T18:07:06.596 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:07:06.819 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T18:07:06.820 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:07:06.820 INFO:tasks.workunit.client.0.trial195.stderr:2' 2026-02-20T18:07:06.820 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-20T18:07:06.820 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-20T18:07:06.820 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-20T18:07:06.820 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=2 2026-02-20T18:07:06.820 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-20T18:07:07.132 INFO:tasks.workunit.client.0.trial195.stderr:noout is set 2026-02-20T18:07:07.143 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-20T18:07:07.143 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 10 = 0 ']' 2026-02-20T18:07:07.143 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:224: set_size: dd if=/dev/urandom bs=10 count=1 of=td/test-erasure-eio/CORRUPT 2026-02-20T18:07:07.144 INFO:tasks.workunit.client.0.trial195.stderr:1+0 records in 2026-02-20T18:07:07.144 INFO:tasks.workunit.client.0.trial195.stderr:1+0 records out 2026-02-20T18:07:07.144 INFO:tasks.workunit.client.0.trial195.stderr:10 bytes copied, 4.6968e-05 s, 213 kB/s 2026-02-20T18:07:07.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 2 obj-size-85808-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:07:07.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-20T18:07:07.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-20T18:07:07.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-20T18:07:07.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-20T18:07:07.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-size-85808-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:07:07.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-20T18:07:07.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-20T18:07:07.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-20T18:07:07.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-20T18:07:07.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-20T18:07:07.145 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:07:07.145 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:07:07.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:07:07.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:07:07.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:07:07.453 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:07:07.453 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-size-85808-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:07:07.454 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-20T18:07:07.454 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-20T18:07:07.454 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-20T18:07:07.454 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-20T18:07:07.454 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-20T18:07:07.454 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-size-85808-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:07:07.600 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-20T18:07:07.600 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-20T18:07:07.600 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-20T18:07:07.601 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-20T18:07:07.601 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-20T18:07:07.601 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-20T18:07:07.601 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:07:07.601 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:07:07.601 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:07:07.601 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:07:07.601 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-20T18:07:07.601 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-20T18:07:07.601 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-20T18:07:07.601 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-20T18:07:07.601 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:07:07.601 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-20T18:07:07.601 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:07:07.601 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:07:07.601 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:07:07.602 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:07:07.602 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:07:07.602 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:07:07.602 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:07:07.602 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:07:07.602 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:07:07.602 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:07:07.602 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:07:07.602 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:07:07.602 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:07:07.602 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:07:07.602 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-20T18:07:07.602 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-20T18:07:07.602 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-20T18:07:07.602 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-20T18:07:07.602 INFO:tasks.workunit.client.0.trial195.stderr:start osd.2 2026-02-20T18:07:07.602 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:07:07.603 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-20T18:07:07.604 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-20T18:07:07.604 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-20T18:07:07.604 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-20T18:07:07.604 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-20T18:07:07.620 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:07.619+0000 7f28a3097900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:07.622 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:07.621+0000 7f28a3097900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:07.623 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:07.622+0000 7f28a3097900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:07.774 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:07.773+0000 7f28a3097900 -1 Falling back to public interface 2026-02-20T18:07:07.831 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-20T18:07:07.831 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:07:07.831 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-20T18:07:07.831 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:07:07.831 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:07:07.831 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:07:07.832 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:07:07.832 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T18:07:07.832 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:07:07.832 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:07:08.057 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:07:08.080 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:08.079+0000 7f28a3097900 -1 osd.2 43 log_to_monitors true 2026-02-20T18:07:09.058 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:07:09.058 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:07:09.058 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:07:09.058 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:07:09.058 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:07:09.058 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:07:09.282 INFO:tasks.workunit.client.0.trial195.stderr:osd.2 up in weight 1 up_from 48 up_thru 36 down_at 45 last_clean_interval [18,43) [v2:127.0.0.1:6818/3281617593,v1:127.0.0.1:6819/3281617593] [v2:127.0.0.1:6820/3281617593,v1:127.0.0.1:6821/3281617593] exists,up 3c193710-aac4-4283-800c-b68162ff83c5 2026-02-20T18:07:09.282 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:07:09.282 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:07:09.282 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:07:09.282 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-20T18:07:09.282 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:07:09.282 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:07:09.282 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:07:09.283 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:07:09.283 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:07:09.283 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:07:09.283 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:07:09.283 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:07:09.283 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:07:09.368 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:07:09.368 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:07:09.368 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:07:09.368 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:07:09.368 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:07:09.369 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:07:09.592 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:07:09.592 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:07:09.592 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:07:09.592 INFO:tasks.workunit.client.0.trial195.stderr:3' 2026-02-20T18:07:09.592 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:07:09.593 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:07:09.593 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:07:09.658 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803787 2026-02-20T18:07:09.658 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803787 2026-02-20T18:07:09.658 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803787' 2026-02-20T18:07:09.658 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:07:09.658 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:07:09.723 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724550 2026-02-20T18:07:09.723 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724550 2026-02-20T18:07:09.723 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803787 1-167503724550' 2026-02-20T18:07:09.723 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:07:09.723 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:07:09.789 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=206158430211 2026-02-20T18:07:09.789 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 206158430211 2026-02-20T18:07:09.789 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803787 1-167503724550 2-206158430211' 2026-02-20T18:07:09.789 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:07:09.789 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:07:09.854 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149706 2026-02-20T18:07:09.854 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149706 2026-02-20T18:07:09.854 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803787 1-167503724550 2-206158430211 3-111669149706' 2026-02-20T18:07:09.854 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:07:09.854 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803787 2026-02-20T18:07:09.855 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:07:09.855 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:07:09.855 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803787 2026-02-20T18:07:09.856 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:07:09.856 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803787 2026-02-20T18:07:09.857 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803787' 2026-02-20T18:07:09.857 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.0 seq 25769803787 2026-02-20T18:07:09.857 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:07:10.078 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803785 -lt 25769803787 2026-02-20T18:07:10.078 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:07:11.080 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:07:11.080 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:07:11.305 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803785 -lt 25769803787 2026-02-20T18:07:11.305 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:07:12.306 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-20T18:07:12.307 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:07:12.534 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803787 -lt 25769803787 2026-02-20T18:07:12.534 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:07:12.534 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-167503724550 2026-02-20T18:07:12.535 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:07:12.535 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:07:12.536 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-167503724550 2026-02-20T18:07:12.536 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:07:12.537 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=167503724550 2026-02-20T18:07:12.537 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 167503724550' 2026-02-20T18:07:12.537 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.1 seq 167503724550 2026-02-20T18:07:12.538 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:07:12.768 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 167503724550 -lt 167503724550 2026-02-20T18:07:12.768 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:07:12.769 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-206158430211 2026-02-20T18:07:12.769 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:07:12.770 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:07:12.770 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-206158430211 2026-02-20T18:07:12.770 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:07:12.771 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=206158430211 2026-02-20T18:07:12.771 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 206158430211' 2026-02-20T18:07:12.771 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.2 seq 206158430211 2026-02-20T18:07:12.772 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:07:12.999 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 206158430211 -lt 206158430211 2026-02-20T18:07:12.999 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:07:13.000 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149706 2026-02-20T18:07:13.000 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:07:13.000 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:07:13.001 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149706 2026-02-20T18:07:13.001 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:07:13.002 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149706 2026-02-20T18:07:13.002 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149706' 2026-02-20T18:07:13.002 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.3 seq 111669149706 2026-02-20T18:07:13.003 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:07:13.236 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149706 -lt 111669149706 2026-02-20T18:07:13.237 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:07:13.237 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:07:13.237 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:07:13.558 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:07:13.558 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:07:13.558 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:07:13.558 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:07:13.558 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:07:13.559 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:07:13.559 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:07:13.559 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:07:13.780 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:07:13.780 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:07:13.781 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:07:13.781 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:07:14.092 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:07:14.092 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:07:14.092 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:07:14.092 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-20T18:07:14.093 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-20T18:07:14.411 INFO:tasks.workunit.client.0.trial195.stderr:noout is unset 2026-02-20T18:07:14.420 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-20T18:07:14.421 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:258: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-85808-1-10 2026-02-20T18:07:14.421 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-20T18:07:14.421 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-20T18:07:14.421 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-85808-1-10 2026-02-20T18:07:14.421 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-20T18:07:14.421 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-20T18:07:14.421 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-85808-1-10 td/test-erasure-eio/COPY 2026-02-20T18:07:14.441 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-20T18:07:14.442 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-20T18:07:14.443 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-20T18:07:14.444 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:344: TEST_rados_get_bad_size_shard_1: rados_get_data_bad_size td/test-erasure-eio 1 0 2026-02-20T18:07:14.444 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-20T18:07:14.444 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-20T18:07:14.444 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=1 2026-02-20T18:07:14.444 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-20T18:07:14.444 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=0 2026-02-20T18:07:14.444 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-20T18:07:14.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=set 2026-02-20T18:07:14.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-20T18:07:14.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-85808-1-0 2026-02-20T18:07:14.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-85808-1-0 2026-02-20T18:07:14.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-20T18:07:14.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-20T18:07:14.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-85808-1-0 2026-02-20T18:07:14.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:07:14.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-20T18:07:14.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:07:14.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-20T18:07:14.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:07:14.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-20T18:07:14.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:07:14.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-20T18:07:14.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-85808-1-0 td/test-erasure-eio/ORIGINAL 2026-02-20T18:07:14.464 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-85808-1-0 td/test-erasure-eio 1 0 set 2026-02-20T18:07:14.464 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-85808-1-0 2026-02-20T18:07:14.464 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-20T18:07:14.464 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-20T18:07:14.464 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-20T18:07:14.464 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-20T18:07:14.464 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-20T18:07:14.464 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=0 2026-02-20T18:07:14.464 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-20T18:07:14.464 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-20T18:07:14.464 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-20T18:07:14.464 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-85808-1-0 2026-02-20T18:07:14.464 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:07:14.464 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-85808-1-0 2026-02-20T18:07:14.464 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-85808-1-0 2026-02-20T18:07:14.465 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:07:14.692 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T18:07:14.692 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:07:14.692 INFO:tasks.workunit.client.0.trial195.stderr:2' 2026-02-20T18:07:14.693 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-20T18:07:14.693 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-20T18:07:14.693 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-20T18:07:14.693 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-20T18:07:14.693 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-20T18:07:14.976 INFO:tasks.workunit.client.0.trial195.stderr:noout is set 2026-02-20T18:07:14.987 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-20T18:07:14.987 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 0 = 0 ']' 2026-02-20T18:07:14.987 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:222: set_size: touch td/test-erasure-eio/CORRUPT 2026-02-20T18:07:14.989 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-85808-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:07:14.989 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-20T18:07:14.989 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-20T18:07:14.989 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-20T18:07:14.989 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-20T18:07:14.989 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-85808-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:07:14.989 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-20T18:07:14.990 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-20T18:07:14.990 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-20T18:07:14.990 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-20T18:07:14.990 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-20T18:07:14.990 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:07:14.990 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:07:14.990 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:07:14.990 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:07:14.990 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:07:15.295 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:07:15.295 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-85808-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:07:15.295 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-20T18:07:15.295 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-20T18:07:15.295 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-20T18:07:15.295 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-20T18:07:15.295 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-20T18:07:15.295 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-85808-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:07:15.425 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-20T18:07:15.425 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-20T18:07:15.425 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-20T18:07:15.426 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-20T18:07:15.426 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-20T18:07:15.426 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-20T18:07:15.426 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:07:15.426 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:07:15.426 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:07:15.426 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:07:15.426 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-20T18:07:15.426 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-20T18:07:15.426 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-20T18:07:15.427 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-20T18:07:15.427 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:07:15.427 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-20T18:07:15.427 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:07:15.427 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:07:15.427 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:07:15.427 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:07:15.427 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:07:15.427 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:07:15.427 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:07:15.427 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:07:15.427 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:07:15.428 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:07:15.428 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:07:15.428 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:07:15.428 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:07:15.428 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:07:15.428 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-20T18:07:15.428 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-20T18:07:15.428 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-20T18:07:15.428 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-20T18:07:15.428 INFO:tasks.workunit.client.0.trial195.stderr:start osd.1 2026-02-20T18:07:15.428 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:07:15.428 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-20T18:07:15.429 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-20T18:07:15.429 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-20T18:07:15.429 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-20T18:07:15.429 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-20T18:07:15.439 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:15.439+0000 7fdf40d60900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:15.441 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:15.440+0000 7fdf40d60900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:15.442 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:15.441+0000 7fdf40d60900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:15.574 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:15.574+0000 7fdf40d60900 -1 Falling back to public interface 2026-02-20T18:07:15.653 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-20T18:07:15.653 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:07:15.653 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-20T18:07:15.654 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:07:15.654 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:07:15.654 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:07:15.654 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:07:15.654 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T18:07:15.654 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:07:15.654 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:07:15.873 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:15.872+0000 7fdf40d60900 -1 osd.1 51 log_to_monitors true 2026-02-20T18:07:15.877 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:07:16.879 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:07:16.879 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:07:16.879 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:07:16.879 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:07:16.880 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:07:16.880 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:07:17.102 INFO:tasks.workunit.client.0.trial195.stderr:osd.1 up in weight 1 up_from 57 up_thru 57 down_at 54 last_clean_interval [39,51) [v2:127.0.0.1:6810/740367737,v1:127.0.0.1:6811/740367737] [v2:127.0.0.1:6812/740367737,v1:127.0.0.1:6813/740367737] exists,up 56e4a9c1-3b40-41fc-b068-854b88a18a1b 2026-02-20T18:07:17.102 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:07:17.103 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:07:17.103 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:07:17.103 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-20T18:07:17.103 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:07:17.103 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:07:17.103 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:07:17.103 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:07:17.104 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:07:17.104 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:07:17.104 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:07:17.104 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:07:17.104 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:07:17.186 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:07:17.186 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:07:17.186 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:07:17.186 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:07:17.186 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:07:17.186 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:07:17.411 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:07:17.411 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:07:17.411 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:07:17.412 INFO:tasks.workunit.client.0.trial195.stderr:3' 2026-02-20T18:07:17.412 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:07:17.412 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:07:17.412 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:07:17.481 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803791 2026-02-20T18:07:17.481 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803791 2026-02-20T18:07:17.482 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803791' 2026-02-20T18:07:17.482 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:07:17.482 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:07:17.548 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=244813135875 2026-02-20T18:07:17.548 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 244813135875 2026-02-20T18:07:17.548 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803791 1-244813135875' 2026-02-20T18:07:17.548 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:07:17.549 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:07:17.614 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=206158430214 2026-02-20T18:07:17.614 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 206158430214 2026-02-20T18:07:17.615 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803791 1-244813135875 2-206158430214' 2026-02-20T18:07:17.615 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:07:17.615 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:07:17.679 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149709 2026-02-20T18:07:17.679 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149709 2026-02-20T18:07:17.679 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803791 1-244813135875 2-206158430214 3-111669149709' 2026-02-20T18:07:17.679 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:07:17.680 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803791 2026-02-20T18:07:17.680 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:07:17.680 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:07:17.682 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803791 2026-02-20T18:07:17.682 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:07:17.683 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803791 2026-02-20T18:07:17.683 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803791' 2026-02-20T18:07:17.683 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.0 seq 25769803791 2026-02-20T18:07:17.684 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:07:17.915 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803791 -lt 25769803791 2026-02-20T18:07:17.915 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:07:17.916 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-244813135875 2026-02-20T18:07:17.916 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:07:17.916 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:07:17.917 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-244813135875 2026-02-20T18:07:17.917 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:07:17.918 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=244813135875 2026-02-20T18:07:17.919 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 244813135875' 2026-02-20T18:07:17.919 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.1 seq 244813135875 2026-02-20T18:07:17.919 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:07:18.143 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 244813135875 -lt 244813135875 2026-02-20T18:07:18.143 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:07:18.144 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-206158430214 2026-02-20T18:07:18.144 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:07:18.144 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:07:18.145 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-206158430214 2026-02-20T18:07:18.145 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:07:18.146 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=206158430214 2026-02-20T18:07:18.146 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 206158430214' 2026-02-20T18:07:18.146 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.2 seq 206158430214 2026-02-20T18:07:18.146 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:07:18.372 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 206158430212 -lt 206158430214 2026-02-20T18:07:18.372 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:07:19.373 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:07:19.374 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:07:19.600 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 206158430212 -lt 206158430214 2026-02-20T18:07:19.600 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:07:20.601 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-20T18:07:20.601 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:07:20.826 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 206158430215 -lt 206158430214 2026-02-20T18:07:20.826 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:07:20.827 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149709 2026-02-20T18:07:20.827 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:07:20.828 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:07:20.828 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149709 2026-02-20T18:07:20.828 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:07:20.829 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149709 2026-02-20T18:07:20.829 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149709' 2026-02-20T18:07:20.829 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.3 seq 111669149709 2026-02-20T18:07:20.830 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:07:21.053 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149709 -lt 111669149709 2026-02-20T18:07:21.054 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:07:21.054 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:07:21.054 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:07:21.366 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:07:21.366 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:07:21.366 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:07:21.366 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:07:21.367 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:07:21.367 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:07:21.367 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:07:21.367 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:07:21.587 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:07:21.587 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:07:21.588 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:07:21.588 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:07:21.899 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:07:21.899 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:07:21.899 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:07:21.899 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-20T18:07:21.899 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-20T18:07:22.215 INFO:tasks.workunit.client.0.trial195.stderr:noout is unset 2026-02-20T18:07:22.226 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-85808-1-0 2026-02-20T18:07:22.226 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-20T18:07:22.226 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-20T18:07:22.226 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-85808-1-0 2026-02-20T18:07:22.226 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-20T18:07:22.227 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-20T18:07:22.227 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-85808-1-0 td/test-erasure-eio/COPY 2026-02-20T18:07:22.247 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-20T18:07:22.248 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-20T18:07:22.249 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 1 + 1 2026-02-20T18:07:22.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=2 2026-02-20T18:07:22.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-85808-1-0 td/test-erasure-eio 2 0 set 2026-02-20T18:07:22.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-85808-1-0 2026-02-20T18:07:22.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-20T18:07:22.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-20T18:07:22.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-20T18:07:22.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=2 2026-02-20T18:07:22.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-20T18:07:22.253 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=0 2026-02-20T18:07:22.253 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-20T18:07:22.253 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-20T18:07:22.253 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-20T18:07:22.253 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-85808-1-0 2026-02-20T18:07:22.253 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:07:22.253 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-85808-1-0 2026-02-20T18:07:22.253 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-85808-1-0 2026-02-20T18:07:22.253 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:07:22.479 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T18:07:22.480 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:07:22.480 INFO:tasks.workunit.client.0.trial195.stderr:2' 2026-02-20T18:07:22.480 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-20T18:07:22.480 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-20T18:07:22.480 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-20T18:07:22.480 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=2 2026-02-20T18:07:22.480 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-20T18:07:22.790 INFO:tasks.workunit.client.0.trial195.stderr:noout is set 2026-02-20T18:07:22.801 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-20T18:07:22.801 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 0 = 0 ']' 2026-02-20T18:07:22.801 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:222: set_size: touch td/test-erasure-eio/CORRUPT 2026-02-20T18:07:22.802 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 2 obj-size-85808-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:07:22.802 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-20T18:07:22.802 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-20T18:07:22.802 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-20T18:07:22.802 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-20T18:07:22.802 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-size-85808-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:07:22.802 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-20T18:07:22.803 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-20T18:07:22.803 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-20T18:07:22.803 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-20T18:07:22.803 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-20T18:07:22.803 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:07:22.803 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:07:22.803 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:07:22.803 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:07:22.803 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:07:23.110 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:07:23.111 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-size-85808-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:07:23.111 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-20T18:07:23.111 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-20T18:07:23.111 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-20T18:07:23.111 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-20T18:07:23.111 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-20T18:07:23.111 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-size-85808-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:07:23.247 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-20T18:07:23.247 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-20T18:07:23.247 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-20T18:07:23.247 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-20T18:07:23.247 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-20T18:07:23.247 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-20T18:07:23.247 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:07:23.247 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:07:23.247 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:07:23.248 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:07:23.248 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-20T18:07:23.248 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-20T18:07:23.248 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-20T18:07:23.248 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-20T18:07:23.248 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:07:23.248 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-20T18:07:23.248 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:07:23.248 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:07:23.248 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:07:23.248 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:07:23.248 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:07:23.248 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:07:23.248 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:07:23.248 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:07:23.248 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:07:23.249 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:07:23.249 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:07:23.249 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:07:23.249 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:07:23.249 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:07:23.249 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-20T18:07:23.249 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-20T18:07:23.249 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-20T18:07:23.249 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-20T18:07:23.249 INFO:tasks.workunit.client.0.trial195.stderr:start osd.2 2026-02-20T18:07:23.249 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:07:23.249 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-20T18:07:23.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-20T18:07:23.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-20T18:07:23.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-20T18:07:23.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-20T18:07:23.264 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:23.264+0000 7efc49a22900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:23.266 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:23.265+0000 7efc49a22900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:23.267 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:23.266+0000 7efc49a22900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:23.406 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:23.404+0000 7efc49a22900 -1 Falling back to public interface 2026-02-20T18:07:23.475 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-20T18:07:23.475 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:07:23.475 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-20T18:07:23.475 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:07:23.475 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:07:23.476 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:07:23.476 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:07:23.476 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T18:07:23.476 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:07:23.476 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:07:23.700 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:07:23.752 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:23.751+0000 7efc49a22900 -1 osd.2 62 log_to_monitors true 2026-02-20T18:07:24.702 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:07:24.702 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:07:24.702 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:07:24.702 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:07:24.702 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:07:24.702 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:07:24.932 INFO:tasks.workunit.client.0.trial195.stderr:osd.2 up in weight 1 up_from 66 up_thru 54 down_at 63 last_clean_interval [48,62) [v2:127.0.0.1:6818/920581102,v1:127.0.0.1:6819/920581102] [v2:127.0.0.1:6820/920581102,v1:127.0.0.1:6821/920581102] exists,up 3c193710-aac4-4283-800c-b68162ff83c5 2026-02-20T18:07:24.932 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:07:24.932 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:07:24.932 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:07:24.932 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-20T18:07:24.932 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:07:24.932 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:07:24.932 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:07:24.932 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:07:24.933 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:07:24.933 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:07:24.933 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:07:24.933 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:07:24.933 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:07:25.018 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:07:25.018 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:07:25.018 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:07:25.018 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:07:25.018 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:07:25.019 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:07:25.243 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:07:25.244 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:07:25.244 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:07:25.244 INFO:tasks.workunit.client.0.trial195.stderr:3' 2026-02-20T18:07:25.244 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:07:25.244 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:07:25.244 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:07:25.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803794 2026-02-20T18:07:25.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803794 2026-02-20T18:07:25.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803794' 2026-02-20T18:07:25.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:07:25.308 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:07:25.373 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=244813135878 2026-02-20T18:07:25.373 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 244813135878 2026-02-20T18:07:25.373 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803794 1-244813135878' 2026-02-20T18:07:25.373 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:07:25.373 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:07:25.439 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=283467841539 2026-02-20T18:07:25.440 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 283467841539 2026-02-20T18:07:25.440 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803794 1-244813135878 2-283467841539' 2026-02-20T18:07:25.440 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:07:25.440 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:07:25.507 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149713 2026-02-20T18:07:25.507 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149713 2026-02-20T18:07:25.508 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803794 1-244813135878 2-283467841539 3-111669149713' 2026-02-20T18:07:25.508 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:07:25.508 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803794 2026-02-20T18:07:25.508 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:07:25.508 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:07:25.509 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803794 2026-02-20T18:07:25.509 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:07:25.510 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803794 2026-02-20T18:07:25.510 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803794' 2026-02-20T18:07:25.510 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.0 seq 25769803794 2026-02-20T18:07:25.511 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:07:25.738 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803794 -lt 25769803794 2026-02-20T18:07:25.738 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:07:25.739 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-244813135878 2026-02-20T18:07:25.739 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:07:25.739 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:07:25.740 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-244813135878 2026-02-20T18:07:25.740 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:07:25.741 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=244813135878 2026-02-20T18:07:25.741 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 244813135878' 2026-02-20T18:07:25.741 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.1 seq 244813135878 2026-02-20T18:07:25.741 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:07:25.964 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 244813135878 -lt 244813135878 2026-02-20T18:07:25.964 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:07:25.965 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-283467841539 2026-02-20T18:07:25.965 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:07:25.966 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:07:25.966 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-283467841539 2026-02-20T18:07:25.966 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:07:25.968 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=283467841539 2026-02-20T18:07:25.968 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 283467841539' 2026-02-20T18:07:25.968 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.2 seq 283467841539 2026-02-20T18:07:25.968 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:07:26.199 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 283467841539 -lt 283467841539 2026-02-20T18:07:26.200 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:07:26.200 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149713 2026-02-20T18:07:26.200 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:07:26.201 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:07:26.201 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149713 2026-02-20T18:07:26.202 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:07:26.203 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149713 2026-02-20T18:07:26.203 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149713' 2026-02-20T18:07:26.203 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.3 seq 111669149713 2026-02-20T18:07:26.203 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:07:26.427 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149713 -lt 111669149713 2026-02-20T18:07:26.428 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:07:26.428 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:07:26.428 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:07:26.745 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:07:26.745 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:07:26.745 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:07:26.745 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:07:26.745 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:07:26.745 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:07:26.745 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:07:26.745 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:07:26.964 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:07:26.964 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:07:26.964 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:07:26.965 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:07:27.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:07:27.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:07:27.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:07:27.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-20T18:07:27.278 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-20T18:07:27.590 INFO:tasks.workunit.client.0.trial195.stderr:noout is unset 2026-02-20T18:07:27.601 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-20T18:07:27.601 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:258: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-85808-1-0 2026-02-20T18:07:27.601 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-20T18:07:27.601 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-20T18:07:27.601 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-85808-1-0 2026-02-20T18:07:27.601 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-20T18:07:27.601 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-20T18:07:27.602 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-85808-1-0 td/test-erasure-eio/COPY 2026-02-20T18:07:27.622 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-20T18:07:27.622 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-20T18:07:27.623 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-20T18:07:27.626 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:345: TEST_rados_get_bad_size_shard_1: rados_get_data_bad_size td/test-erasure-eio 1 256 add 2026-02-20T18:07:27.626 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-20T18:07:27.626 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-20T18:07:27.626 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=1 2026-02-20T18:07:27.626 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-20T18:07:27.626 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=256 2026-02-20T18:07:27.626 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-20T18:07:27.626 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=add 2026-02-20T18:07:27.627 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-20T18:07:27.627 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-85808-1-256 2026-02-20T18:07:27.627 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-85808-1-256 2026-02-20T18:07:27.627 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-20T18:07:27.627 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-20T18:07:27.627 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-85808-1-256 2026-02-20T18:07:27.627 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:07:27.627 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-20T18:07:27.627 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:07:27.627 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-20T18:07:27.627 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:07:27.627 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-20T18:07:27.627 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:07:27.627 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-20T18:07:27.628 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-85808-1-256 td/test-erasure-eio/ORIGINAL 2026-02-20T18:07:27.645 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-85808-1-256 td/test-erasure-eio 1 256 add 2026-02-20T18:07:27.645 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-85808-1-256 2026-02-20T18:07:27.645 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-20T18:07:27.645 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-20T18:07:27.645 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-20T18:07:27.645 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-20T18:07:27.645 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-20T18:07:27.645 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=256 2026-02-20T18:07:27.645 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-20T18:07:27.645 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=add 2026-02-20T18:07:27.645 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-20T18:07:27.645 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-85808-1-256 2026-02-20T18:07:27.645 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:07:27.645 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-85808-1-256 2026-02-20T18:07:27.645 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-85808-1-256 2026-02-20T18:07:27.645 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:07:27.875 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T18:07:27.875 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:07:27.875 INFO:tasks.workunit.client.0.trial195.stderr:2' 2026-02-20T18:07:27.876 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-20T18:07:27.876 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-20T18:07:27.876 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-20T18:07:27.876 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-20T18:07:27.876 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-20T18:07:28.157 INFO:tasks.workunit.client.0.trial195.stderr:noout is set 2026-02-20T18:07:28.168 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' add = add ']' 2026-02-20T18:07:28.169 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-85808-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:07:28.169 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-20T18:07:28.169 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-20T18:07:28.169 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-20T18:07:28.169 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-20T18:07:28.169 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-85808-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:07:28.169 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-20T18:07:28.169 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-20T18:07:28.169 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-20T18:07:28.170 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-20T18:07:28.170 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-20T18:07:28.170 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:07:28.170 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:07:28.170 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:07:28.170 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:07:28.170 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:07:28.475 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:07:28.475 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-85808-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:07:28.476 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-20T18:07:28.476 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-20T18:07:28.476 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-20T18:07:28.476 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-20T18:07:28.476 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-20T18:07:28.476 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-85808-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:07:28.559 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-20T18:07:28.559 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-20T18:07:28.559 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-20T18:07:28.559 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-20T18:07:28.559 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-20T18:07:28.559 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-20T18:07:28.559 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:07:28.559 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:07:28.559 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:07:28.559 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:07:28.559 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-20T18:07:28.559 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-20T18:07:28.559 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-20T18:07:28.560 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-20T18:07:28.560 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:07:28.560 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-20T18:07:28.560 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:07:28.560 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:07:28.560 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:07:28.560 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:07:28.560 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:07:28.560 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:07:28.560 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:07:28.560 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:07:28.560 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:07:28.560 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:07:28.560 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:07:28.560 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:07:28.560 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:07:28.560 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:07:28.560 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-20T18:07:28.560 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-20T18:07:28.560 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-20T18:07:28.560 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-20T18:07:28.560 INFO:tasks.workunit.client.0.trial195.stderr:start osd.1 2026-02-20T18:07:28.560 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:07:28.560 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-20T18:07:28.560 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-20T18:07:28.560 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-20T18:07:28.560 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-20T18:07:28.560 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-20T18:07:28.571 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:28.571+0000 7fb6d7491900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:28.573 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:28.572+0000 7fb6d7491900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:28.574 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:28.573+0000 7fb6d7491900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:28.721 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:28.721+0000 7fb6d7491900 -1 Falling back to public interface 2026-02-20T18:07:28.780 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-20T18:07:28.780 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:07:28.780 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-20T18:07:28.781 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:07:28.781 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:07:28.781 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:07:28.781 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:07:28.781 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T18:07:28.781 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:07:28.781 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:07:29.008 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:07:29.042 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:29.041+0000 7fb6d7491900 -1 osd.1 71 log_to_monitors true 2026-02-20T18:07:30.009 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:07:30.009 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:07:30.009 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:07:30.009 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:07:30.010 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:07:30.010 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:07:30.236 INFO:tasks.workunit.client.0.trial195.stderr:osd.1 up in weight 1 up_from 75 up_thru 75 down_at 72 last_clean_interval [57,71) [v2:127.0.0.1:6810/3618531563,v1:127.0.0.1:6811/3618531563] [v2:127.0.0.1:6812/3618531563,v1:127.0.0.1:6813/3618531563] exists,up 56e4a9c1-3b40-41fc-b068-854b88a18a1b 2026-02-20T18:07:30.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:07:30.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:07:30.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:07:30.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-20T18:07:30.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:07:30.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:07:30.238 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:07:30.238 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:07:30.238 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:07:30.238 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:07:30.239 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:07:30.239 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:07:30.239 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:07:30.321 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:07:30.321 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:07:30.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:07:30.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:07:30.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:07:30.322 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:07:30.549 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:07:30.550 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:07:30.550 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:07:30.550 INFO:tasks.workunit.client.0.trial195.stderr:3' 2026-02-20T18:07:30.550 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:07:30.550 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:07:30.550 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:07:30.614 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803797 2026-02-20T18:07:30.614 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803797 2026-02-20T18:07:30.614 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803797' 2026-02-20T18:07:30.614 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:07:30.614 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:07:30.679 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=322122547203 2026-02-20T18:07:30.679 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 322122547203 2026-02-20T18:07:30.679 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803797 1-322122547203' 2026-02-20T18:07:30.679 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:07:30.679 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:07:30.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=283467841542 2026-02-20T18:07:30.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 283467841542 2026-02-20T18:07:30.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803797 1-322122547203 2-283467841542' 2026-02-20T18:07:30.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:07:30.745 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:07:30.807 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149716 2026-02-20T18:07:30.808 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149716 2026-02-20T18:07:30.808 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803797 1-322122547203 2-283467841542 3-111669149716' 2026-02-20T18:07:30.808 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:07:30.808 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803797 2026-02-20T18:07:30.808 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:07:30.809 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:07:30.809 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803797 2026-02-20T18:07:30.810 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:07:30.811 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803797 2026-02-20T18:07:30.811 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803797' 2026-02-20T18:07:30.811 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.0 seq 25769803797 2026-02-20T18:07:30.811 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:07:31.037 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803795 -lt 25769803797 2026-02-20T18:07:31.037 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:07:32.039 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:07:32.039 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:07:32.273 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803797 -lt 25769803797 2026-02-20T18:07:32.273 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:07:32.273 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-322122547203 2026-02-20T18:07:32.273 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:07:32.274 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:07:32.276 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-322122547203 2026-02-20T18:07:32.276 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:07:32.276 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=322122547203 2026-02-20T18:07:32.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 322122547203' 2026-02-20T18:07:32.277 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.1 seq 322122547203 2026-02-20T18:07:32.277 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:07:32.504 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 322122547203 -lt 322122547203 2026-02-20T18:07:32.504 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:07:32.505 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-283467841542 2026-02-20T18:07:32.505 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:07:32.506 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:07:32.506 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-283467841542 2026-02-20T18:07:32.506 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:07:32.507 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=283467841542 2026-02-20T18:07:32.507 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 283467841542' 2026-02-20T18:07:32.508 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.2 seq 283467841542 2026-02-20T18:07:32.508 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:07:32.730 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 283467841542 -lt 283467841542 2026-02-20T18:07:32.730 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:07:32.731 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149716 2026-02-20T18:07:32.731 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:07:32.731 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:07:32.732 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149716 2026-02-20T18:07:32.732 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:07:32.733 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149716 2026-02-20T18:07:32.733 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149716' 2026-02-20T18:07:32.733 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.3 seq 111669149716 2026-02-20T18:07:32.734 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:07:32.955 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149716 -lt 111669149716 2026-02-20T18:07:32.956 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:07:32.956 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:07:32.956 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:07:33.269 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:07:33.269 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:07:33.270 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:07:33.270 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:07:33.270 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:07:33.270 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:07:33.270 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:07:33.270 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:07:33.489 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:07:33.489 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:07:33.489 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:07:33.490 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:07:33.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:07:33.799 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:07:33.799 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:07:33.799 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:219: set_size: dd if=/dev/urandom bs=256 count=1 2026-02-20T18:07:33.801 INFO:tasks.workunit.client.0.trial195.stderr:1+0 records in 2026-02-20T18:07:33.801 INFO:tasks.workunit.client.0.trial195.stderr:1+0 records out 2026-02-20T18:07:33.801 INFO:tasks.workunit.client.0.trial195.stderr:256 bytes copied, 2.1279e-05 s, 12.0 MB/s 2026-02-20T18:07:33.801 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-85808-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:07:33.801 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-20T18:07:33.801 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-20T18:07:33.801 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-20T18:07:33.801 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-20T18:07:33.801 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-85808-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:07:33.801 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-20T18:07:33.801 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-20T18:07:33.801 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-20T18:07:33.801 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-20T18:07:33.801 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-20T18:07:33.802 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:07:33.802 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:07:33.802 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:07:33.802 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:07:33.802 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:07:34.107 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:07:34.107 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-85808-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:07:34.108 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-20T18:07:34.108 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-20T18:07:34.108 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-20T18:07:34.108 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-20T18:07:34.108 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-20T18:07:34.108 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-85808-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:07:34.246 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-20T18:07:34.247 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-20T18:07:34.247 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-20T18:07:34.247 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-20T18:07:34.247 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-20T18:07:34.247 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-20T18:07:34.247 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:07:34.247 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:07:34.247 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:07:34.247 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:07:34.247 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-20T18:07:34.247 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-20T18:07:34.248 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-20T18:07:34.248 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-20T18:07:34.248 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:07:34.248 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-20T18:07:34.248 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:07:34.248 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:07:34.248 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:07:34.248 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:07:34.248 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:07:34.248 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:07:34.248 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:07:34.248 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:07:34.248 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:07:34.249 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:07:34.249 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:07:34.249 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:07:34.249 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:07:34.249 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:07:34.249 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-20T18:07:34.249 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-20T18:07:34.249 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-20T18:07:34.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-20T18:07:34.250 INFO:tasks.workunit.client.0.trial195.stderr:start osd.1 2026-02-20T18:07:34.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:07:34.250 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-20T18:07:34.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-20T18:07:34.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-20T18:07:34.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-20T18:07:34.251 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-20T18:07:34.263 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:34.261+0000 7fec80b9f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:34.264 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:34.263+0000 7fec80b9f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:34.265 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:34.263+0000 7fec80b9f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:34.386 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:34.383+0000 7fec80b9f900 -1 Falling back to public interface 2026-02-20T18:07:34.474 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-20T18:07:34.474 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:07:34.474 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-20T18:07:34.474 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:07:34.474 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:07:34.474 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:07:34.474 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:07:34.475 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T18:07:34.475 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:07:34.475 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:07:34.701 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:07:34.764 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:34.762+0000 7fec80b9f900 -1 osd.1 76 log_to_monitors true 2026-02-20T18:07:35.703 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:07:35.703 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:07:35.704 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:07:35.704 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:07:35.704 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:07:35.704 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:07:35.929 INFO:tasks.workunit.client.0.trial195.stderr:osd.1 up in weight 1 up_from 80 up_thru 80 down_at 77 last_clean_interval [75,76) [v2:127.0.0.1:6810/2834113136,v1:127.0.0.1:6811/2834113136] [v2:127.0.0.1:6812/2834113136,v1:127.0.0.1:6813/2834113136] exists,up 56e4a9c1-3b40-41fc-b068-854b88a18a1b 2026-02-20T18:07:35.929 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:07:35.929 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:07:35.929 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:07:35.929 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-20T18:07:35.929 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:07:35.929 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:07:35.929 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:07:35.929 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:07:35.930 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:07:35.930 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:07:35.930 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:07:35.930 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:07:35.930 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:07:36.017 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:07:36.018 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:07:36.018 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:07:36.018 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:07:36.018 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:07:36.018 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:07:36.242 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:07:36.242 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:07:36.242 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:07:36.242 INFO:tasks.workunit.client.0.trial195.stderr:3' 2026-02-20T18:07:36.242 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:07:36.242 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:07:36.242 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:07:36.306 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803800 2026-02-20T18:07:36.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803800 2026-02-20T18:07:36.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803800' 2026-02-20T18:07:36.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:07:36.307 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:07:36.371 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=343597383683 2026-02-20T18:07:36.371 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 343597383683 2026-02-20T18:07:36.371 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803800 1-343597383683' 2026-02-20T18:07:36.371 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:07:36.371 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:07:36.434 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=283467841545 2026-02-20T18:07:36.434 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 283467841545 2026-02-20T18:07:36.434 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803800 1-343597383683 2-283467841545' 2026-02-20T18:07:36.434 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:07:36.434 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:07:36.499 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149719 2026-02-20T18:07:36.500 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149719 2026-02-20T18:07:36.500 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803800 1-343597383683 2-283467841545 3-111669149719' 2026-02-20T18:07:36.500 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:07:36.500 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803800 2026-02-20T18:07:36.500 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:07:36.500 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:07:36.501 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803800 2026-02-20T18:07:36.501 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:07:36.502 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803800 2026-02-20T18:07:36.502 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803800' 2026-02-20T18:07:36.502 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.0 seq 25769803800 2026-02-20T18:07:36.503 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:07:36.726 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803798 -lt 25769803800 2026-02-20T18:07:36.726 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:07:37.727 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:07:37.728 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:07:37.952 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803801 -lt 25769803800 2026-02-20T18:07:37.952 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:07:37.953 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-343597383683 2026-02-20T18:07:37.953 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:07:37.955 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:07:37.956 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-343597383683 2026-02-20T18:07:37.956 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:07:37.957 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=343597383683 2026-02-20T18:07:37.958 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 343597383683' 2026-02-20T18:07:37.958 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.1 seq 343597383683 2026-02-20T18:07:37.958 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:07:38.180 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 343597383683 -lt 343597383683 2026-02-20T18:07:38.180 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:07:38.181 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-283467841545 2026-02-20T18:07:38.181 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:07:38.182 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:07:38.182 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:07:38.182 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-283467841545 2026-02-20T18:07:38.183 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=283467841545 2026-02-20T18:07:38.183 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 283467841545' 2026-02-20T18:07:38.183 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.2 seq 283467841545 2026-02-20T18:07:38.184 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:07:38.412 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 283467841545 -lt 283467841545 2026-02-20T18:07:38.412 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:07:38.412 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149719 2026-02-20T18:07:38.412 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:07:38.413 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:07:38.414 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149719 2026-02-20T18:07:38.414 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:07:38.415 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149719 2026-02-20T18:07:38.415 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149719' 2026-02-20T18:07:38.415 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.3 seq 111669149719 2026-02-20T18:07:38.415 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:07:38.646 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149719 -lt 111669149719 2026-02-20T18:07:38.646 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:07:38.646 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:07:38.646 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:07:38.962 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:07:38.962 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:07:38.962 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:07:38.962 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:07:38.963 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:07:38.963 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:07:38.963 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:07:38.963 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:07:39.181 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:07:39.181 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:07:39.181 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:07:39.182 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:07:39.493 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:07:39.494 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:07:39.494 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:07:39.494 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-20T18:07:39.494 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-20T18:07:39.834 INFO:tasks.workunit.client.0.trial195.stderr:noout is unset 2026-02-20T18:07:39.844 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-85808-1-256 2026-02-20T18:07:39.844 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-20T18:07:39.844 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-20T18:07:39.844 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-85808-1-256 2026-02-20T18:07:39.844 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-20T18:07:39.844 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-20T18:07:39.844 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-85808-1-256 td/test-erasure-eio/COPY 2026-02-20T18:07:39.865 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-20T18:07:39.866 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-20T18:07:39.867 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 1 + 1 2026-02-20T18:07:39.869 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=2 2026-02-20T18:07:39.869 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-85808-1-256 td/test-erasure-eio 2 256 add 2026-02-20T18:07:39.869 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-85808-1-256 2026-02-20T18:07:39.870 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-20T18:07:39.870 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-20T18:07:39.870 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-20T18:07:39.870 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=2 2026-02-20T18:07:39.870 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-20T18:07:39.870 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=256 2026-02-20T18:07:39.870 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-20T18:07:39.870 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=add 2026-02-20T18:07:39.870 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-20T18:07:39.871 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-85808-1-256 2026-02-20T18:07:39.871 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:07:39.871 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-85808-1-256 2026-02-20T18:07:39.871 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-85808-1-256 2026-02-20T18:07:39.871 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:07:40.093 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T18:07:40.093 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:07:40.094 INFO:tasks.workunit.client.0.trial195.stderr:2' 2026-02-20T18:07:40.094 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-20T18:07:40.094 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-20T18:07:40.094 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-20T18:07:40.094 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=2 2026-02-20T18:07:40.094 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-20T18:07:40.409 INFO:tasks.workunit.client.0.trial195.stderr:noout is set 2026-02-20T18:07:40.420 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' add = add ']' 2026-02-20T18:07:40.421 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: objectstore_tool td/test-erasure-eio 2 obj-size-85808-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:07:40.421 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-20T18:07:40.421 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-20T18:07:40.421 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-20T18:07:40.421 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-20T18:07:40.421 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-size-85808-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:07:40.421 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-20T18:07:40.421 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-20T18:07:40.421 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-20T18:07:40.421 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-20T18:07:40.421 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-20T18:07:40.421 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:07:40.421 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:07:40.421 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:07:40.421 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:07:40.422 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:07:40.730 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:07:40.730 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-size-85808-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:07:40.730 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-20T18:07:40.730 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-20T18:07:40.731 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-20T18:07:40.731 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-20T18:07:40.731 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-20T18:07:40.731 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-size-85808-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:07:40.818 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-20T18:07:40.819 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-20T18:07:40.819 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-20T18:07:40.819 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-20T18:07:40.819 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-20T18:07:40.819 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-20T18:07:40.820 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:07:40.820 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:07:40.820 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:07:40.820 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:07:40.820 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-20T18:07:40.820 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-20T18:07:40.820 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-20T18:07:40.821 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-20T18:07:40.821 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:07:40.821 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-20T18:07:40.821 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:07:40.821 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:07:40.821 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:07:40.822 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:07:40.822 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:07:40.822 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:07:40.822 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:07:40.822 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:07:40.822 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:07:40.822 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:07:40.823 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:07:40.823 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:07:40.823 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:07:40.823 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:07:40.823 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-20T18:07:40.823 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-20T18:07:40.824 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-20T18:07:40.824 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-20T18:07:40.824 INFO:tasks.workunit.client.0.trial195.stderr:start osd.2 2026-02-20T18:07:40.824 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:07:40.824 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-20T18:07:40.824 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-20T18:07:40.825 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-20T18:07:40.825 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-20T18:07:40.825 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-20T18:07:40.832 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:40.829+0000 7fe033b79900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:40.833 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:40.831+0000 7fe033b79900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:40.834 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:40.832+0000 7fe033b79900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:40.974 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:40.972+0000 7fe033b79900 -1 Falling back to public interface 2026-02-20T18:07:41.042 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-20T18:07:41.043 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:07:41.043 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-20T18:07:41.043 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:07:41.043 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:07:41.043 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:07:41.043 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:07:41.043 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T18:07:41.043 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:07:41.044 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:07:41.267 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:07:41.293 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:41.291+0000 7fe033b79900 -1 osd.2 85 log_to_monitors true 2026-02-20T18:07:42.269 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:07:42.269 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:07:42.269 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:07:42.269 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:07:42.269 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:07:42.270 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:07:42.492 INFO:tasks.workunit.client.0.trial195.stderr:osd.2 up in weight 1 up_from 89 up_thru 77 down_at 86 last_clean_interval [66,85) [v2:127.0.0.1:6818/2442274543,v1:127.0.0.1:6819/2442274543] [v2:127.0.0.1:6820/2442274543,v1:127.0.0.1:6821/2442274543] exists,up 3c193710-aac4-4283-800c-b68162ff83c5 2026-02-20T18:07:42.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:07:42.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:07:42.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:07:42.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-20T18:07:42.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:07:42.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:07:42.493 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:07:42.493 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:07:42.493 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:07:42.493 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:07:42.493 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:07:42.493 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:07:42.493 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:07:42.577 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:07:42.577 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:07:42.577 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:07:42.577 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:07:42.577 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:07:42.577 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:07:42.804 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:07:42.804 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:07:42.804 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:07:42.804 INFO:tasks.workunit.client.0.trial195.stderr:3' 2026-02-20T18:07:42.804 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:07:42.804 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:07:42.804 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:07:42.868 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803804 2026-02-20T18:07:42.868 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803804 2026-02-20T18:07:42.868 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803804' 2026-02-20T18:07:42.868 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:07:42.868 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:07:42.931 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=343597383686 2026-02-20T18:07:42.931 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 343597383686 2026-02-20T18:07:42.932 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803804 1-343597383686' 2026-02-20T18:07:42.932 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:07:42.932 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:07:42.994 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=382252089347 2026-02-20T18:07:42.994 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 382252089347 2026-02-20T18:07:42.994 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803804 1-343597383686 2-382252089347' 2026-02-20T18:07:42.994 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:07:42.994 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:07:43.062 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149722 2026-02-20T18:07:43.063 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149722 2026-02-20T18:07:43.063 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803804 1-343597383686 2-382252089347 3-111669149722' 2026-02-20T18:07:43.063 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:07:43.063 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803804 2026-02-20T18:07:43.063 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:07:43.064 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:07:43.065 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803804 2026-02-20T18:07:43.065 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:07:43.065 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803804 2026-02-20T18:07:43.066 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803804' 2026-02-20T18:07:43.066 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.0 seq 25769803804 2026-02-20T18:07:43.066 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:07:43.290 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803801 -lt 25769803804 2026-02-20T18:07:43.290 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:07:44.291 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:07:44.292 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:07:44.523 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803804 -lt 25769803804 2026-02-20T18:07:44.523 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:07:44.524 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-343597383686 2026-02-20T18:07:44.524 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:07:44.524 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:07:44.525 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-343597383686 2026-02-20T18:07:44.525 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:07:44.527 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=343597383686 2026-02-20T18:07:44.527 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 343597383686' 2026-02-20T18:07:44.527 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.1 seq 343597383686 2026-02-20T18:07:44.527 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:07:44.756 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 343597383686 -lt 343597383686 2026-02-20T18:07:44.756 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:07:44.756 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-382252089347 2026-02-20T18:07:44.756 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:07:44.757 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:07:44.758 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-382252089347 2026-02-20T18:07:44.758 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:07:44.759 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=382252089347 2026-02-20T18:07:44.760 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 382252089347' 2026-02-20T18:07:44.760 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.2 seq 382252089347 2026-02-20T18:07:44.760 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:07:44.989 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 382252089347 -lt 382252089347 2026-02-20T18:07:44.990 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:07:44.990 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149722 2026-02-20T18:07:44.990 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:07:44.991 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:07:44.991 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149722 2026-02-20T18:07:44.991 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:07:44.993 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149722 2026-02-20T18:07:44.993 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149722' 2026-02-20T18:07:44.993 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.3 seq 111669149722 2026-02-20T18:07:44.993 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:07:45.216 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149722 -lt 111669149722 2026-02-20T18:07:45.217 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:07:45.217 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:07:45.217 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:07:45.532 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:07:45.533 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:07:45.533 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:07:45.533 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:07:45.533 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:07:45.533 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:07:45.533 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:07:45.533 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:07:45.754 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:07:45.754 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:07:45.754 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:07:45.755 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:07:46.068 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:07:46.069 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:07:46.069 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:07:46.069 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:219: set_size: dd if=/dev/urandom bs=256 count=1 2026-02-20T18:07:46.070 INFO:tasks.workunit.client.0.trial195.stderr:1+0 records in 2026-02-20T18:07:46.070 INFO:tasks.workunit.client.0.trial195.stderr:1+0 records out 2026-02-20T18:07:46.070 INFO:tasks.workunit.client.0.trial195.stderr:256 bytes copied, 2.0899e-05 s, 12.2 MB/s 2026-02-20T18:07:46.070 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 2 obj-size-85808-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:07:46.070 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-20T18:07:46.070 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-20T18:07:46.070 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-20T18:07:46.070 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-20T18:07:46.070 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-size-85808-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:07:46.070 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-20T18:07:46.071 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-20T18:07:46.071 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-20T18:07:46.071 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-20T18:07:46.071 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-20T18:07:46.071 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:07:46.071 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:07:46.071 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:07:46.071 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:07:46.071 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:07:46.176 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:07:46.176 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-size-85808-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:07:46.176 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-20T18:07:46.176 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-20T18:07:46.176 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-20T18:07:46.176 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-20T18:07:46.176 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-20T18:07:46.176 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-size-85808-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-20T18:07:46.317 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-20T18:07:46.317 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-20T18:07:46.317 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-20T18:07:46.317 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-20T18:07:46.317 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-20T18:07:46.317 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-20T18:07:46.317 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:07:46.317 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:07:46.317 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:07:46.318 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:07:46.318 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-20T18:07:46.318 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-20T18:07:46.318 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-20T18:07:46.318 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-20T18:07:46.318 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:07:46.318 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-20T18:07:46.318 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:07:46.318 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:07:46.318 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:07:46.318 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:07:46.318 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:07:46.318 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:07:46.318 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:07:46.319 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:07:46.319 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:07:46.319 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:07:46.319 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:07:46.319 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:07:46.319 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:07:46.319 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:07:46.319 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-20T18:07:46.319 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-20T18:07:46.319 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-20T18:07:46.319 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-20T18:07:46.319 INFO:tasks.workunit.client.0.trial195.stderr:start osd.2 2026-02-20T18:07:46.319 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:07:46.320 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-20T18:07:46.320 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-20T18:07:46.320 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-20T18:07:46.320 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-20T18:07:46.321 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-20T18:07:46.333 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:46.331+0000 7faa9bb86900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:46.334 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:46.332+0000 7faa9bb86900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:46.335 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:46.333+0000 7faa9bb86900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:46.463 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:46.461+0000 7faa9bb86900 -1 Falling back to public interface 2026-02-20T18:07:46.544 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-20T18:07:46.544 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:07:46.544 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-20T18:07:46.544 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:07:46.544 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:07:46.544 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:07:46.544 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:07:46.545 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T18:07:46.545 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:07:46.545 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:07:46.769 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:07:46.802 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:46.800+0000 7faa9bb86900 -1 osd.2 90 log_to_monitors true 2026-02-20T18:07:47.770 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:07:47.770 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:07:47.770 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:07:47.770 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:07:47.770 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:07:47.770 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:07:47.996 INFO:tasks.workunit.client.0.trial195.stderr:osd.2 up in weight 1 up_from 94 up_thru 77 down_at 91 last_clean_interval [89,90) [v2:127.0.0.1:6818/1886200994,v1:127.0.0.1:6819/1886200994] [v2:127.0.0.1:6820/1886200994,v1:127.0.0.1:6821/1886200994] exists,up 3c193710-aac4-4283-800c-b68162ff83c5 2026-02-20T18:07:47.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:07:47.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:07:47.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:07:47.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-20T18:07:47.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:07:47.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:07:47.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:07:47.996 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:07:47.997 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:07:47.997 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:07:47.997 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:07:47.998 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:07:47.998 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:07:48.083 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:07:48.083 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:07:48.083 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:07:48.083 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:07:48.083 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:07:48.084 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:07:48.315 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:07:48.315 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:07:48.315 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:07:48.316 INFO:tasks.workunit.client.0.trial195.stderr:3' 2026-02-20T18:07:48.316 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:07:48.316 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:07:48.316 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:07:48.381 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803807 2026-02-20T18:07:48.381 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803807 2026-02-20T18:07:48.381 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803807' 2026-02-20T18:07:48.381 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:07:48.381 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:07:48.446 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=343597383689 2026-02-20T18:07:48.446 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 343597383689 2026-02-20T18:07:48.446 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803807 1-343597383689' 2026-02-20T18:07:48.446 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:07:48.446 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:07:48.510 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=403726925827 2026-02-20T18:07:48.510 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 403726925827 2026-02-20T18:07:48.510 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803807 1-343597383689 2-403726925827' 2026-02-20T18:07:48.510 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:07:48.510 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:07:48.576 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149725 2026-02-20T18:07:48.577 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149725 2026-02-20T18:07:48.577 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803807 1-343597383689 2-403726925827 3-111669149725' 2026-02-20T18:07:48.577 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:07:48.577 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803807 2026-02-20T18:07:48.578 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:07:48.578 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:07:48.578 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803807 2026-02-20T18:07:48.579 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:07:48.579 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803807 2026-02-20T18:07:48.579 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803807' 2026-02-20T18:07:48.580 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.0 seq 25769803807 2026-02-20T18:07:48.580 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:07:48.803 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803805 -lt 25769803807 2026-02-20T18:07:48.804 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:07:49.805 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:07:49.805 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:07:50.041 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803807 -lt 25769803807 2026-02-20T18:07:50.041 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:07:50.041 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-343597383689 2026-02-20T18:07:50.041 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:07:50.042 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:07:50.043 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-343597383689 2026-02-20T18:07:50.043 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:07:50.044 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=343597383689 2026-02-20T18:07:50.044 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 343597383689' 2026-02-20T18:07:50.044 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.1 seq 343597383689 2026-02-20T18:07:50.044 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:07:50.269 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 343597383689 -lt 343597383689 2026-02-20T18:07:50.269 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:07:50.270 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-403726925827 2026-02-20T18:07:50.270 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:07:50.270 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:07:50.271 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-403726925827 2026-02-20T18:07:50.271 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:07:50.272 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=403726925827 2026-02-20T18:07:50.272 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 403726925827' 2026-02-20T18:07:50.272 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.2 seq 403726925827 2026-02-20T18:07:50.273 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:07:50.495 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 403726925827 -lt 403726925827 2026-02-20T18:07:50.495 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:07:50.496 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149725 2026-02-20T18:07:50.496 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:07:50.496 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:07:50.497 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149725 2026-02-20T18:07:50.497 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:07:50.498 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149725 2026-02-20T18:07:50.498 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149725' 2026-02-20T18:07:50.498 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.3 seq 111669149725 2026-02-20T18:07:50.499 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:07:50.721 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149725 -lt 111669149725 2026-02-20T18:07:50.721 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:07:50.722 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:07:50.722 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:07:51.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:07:51.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:07:51.036 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:07:51.036 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:07:51.036 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:07:51.036 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:07:51.037 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:07:51.037 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:07:51.255 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:07:51.255 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:07:51.256 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:07:51.256 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:07:51.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:07:51.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:07:51.568 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:07:51.568 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-20T18:07:51.568 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-20T18:07:51.910 INFO:tasks.workunit.client.0.trial195.stderr:noout is unset 2026-02-20T18:07:51.921 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-20T18:07:51.921 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:258: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-85808-1-256 2026-02-20T18:07:51.921 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-20T18:07:51.921 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-20T18:07:51.921 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-85808-1-256 2026-02-20T18:07:51.921 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-20T18:07:51.921 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-20T18:07:51.921 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-85808-1-256 td/test-erasure-eio/COPY 2026-02-20T18:07:51.942 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-20T18:07:51.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-20T18:07:51.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-20T18:07:51.944 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:346: TEST_rados_get_bad_size_shard_1: delete_erasure_coded_pool pool-jerasure 2026-02-20T18:07:51.944 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-20T18:07:51.944 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-20T18:07:52.172 INFO:tasks.workunit.client.0.trial195.stderr:pool 'pool-jerasure' does not exist 2026-02-20T18:07:52.180 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-20T18:07:52.459 INFO:tasks.workunit.client.0.trial195.stderr:erasure-code-profile myprofile does not exist 2026-02-20T18:07:52.467 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-20T18:07:52.467 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-20T18:07:52.468 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T18:07:52.468 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-20T18:07:52.468 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:07:52.468 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:07:52.468 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:07:52.468 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:07:52.468 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:07:52.577 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:07:52.577 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T18:07:52.578 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T18:07:52.578 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T18:07:52.580 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T18:07:52.580 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T18:07:52.581 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T18:07:52.582 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:07:52.582 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T18:07:52.582 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T18:07:52.583 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:07:52.583 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T18:07:52.584 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T18:07:52.584 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-20T18:07:52.600 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T18:07:52.600 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:07:52.600 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:07:52.600 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.85808 2026-02-20T18:07:52.602 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T18:07:52.602 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T18:07:52.602 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-20T18:07:52.603 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-20T18:07:52.603 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-20T18:07:52.603 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-20T18:07:52.603 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-20T18:07:52.603 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T18:07:52.603 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-20T18:07:52.603 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:07:52.603 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:07:52.603 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:07:52.603 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:07:52.603 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:07:52.604 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:07:52.604 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T18:07:52.605 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T18:07:52.605 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T18:07:52.606 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T18:07:52.606 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T18:07:52.606 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T18:07:52.607 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:07:52.607 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T18:07:52.608 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T18:07:52.608 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:07:52.608 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T18:07:52.609 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T18:07:52.609 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-20T18:07:52.610 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T18:07:52.610 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:07:52.610 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:07:52.611 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.85808 2026-02-20T18:07:52.611 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T18:07:52.611 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T18:07:52.611 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-20T18:07:52.613 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-20T18:07:52.613 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:07:52.613 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:07:52.613 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.85808 2026-02-20T18:07:52.614 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-20T18:07:52.615 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-20T18:07:52.615 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-20T18:07:52.615 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-20T18:07:52.615 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-20T18:07:52.616 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-20T18:07:52.616 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-20T18:07:52.616 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-20T18:07:52.616 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-20T18:07:52.616 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-20T18:07:52.616 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-20T18:07:52.636 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-20T18:07:52.636 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:07:52.637 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:07:52.637 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:07:52.637 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:07:52.637 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:07:52.637 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:07:52.637 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-20T18:07:52.662 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-20T18:07:52.664 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-20T18:07:52.664 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T18:07:52.664 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T18:07:52.664 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-20T18:07:52.665 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-20T18:07:52.665 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T18:07:52.665 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:07:52.665 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:07:52.665 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:07:52.665 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:07:52.665 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:07:52.665 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:07:52.665 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T18:07:52.665 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.85808/ceph-mon.a.asok config get fsid 2026-02-20T18:07:52.713 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-20T18:07:52.714 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T18:07:52.714 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T18:07:52.714 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-20T18:07:52.714 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-20T18:07:52.714 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T18:07:52.714 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:07:52.714 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:07:52.714 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:07:52.714 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:07:52.714 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:07:52.714 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:07:52.715 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T18:07:52.715 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.85808/ceph-mon.a.asok config get mon_host 2026-02-20T18:07:52.760 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-20T18:07:52.760 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-20T18:07:52.760 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-20T18:07:52.760 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-20T18:07:52.760 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-20T18:07:52.760 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-20T18:07:52.760 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-20T18:07:52.861 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-20T18:07:52.861 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:07:52.861 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:07:52.861 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:07:52.861 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:07:52.861 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:07:52.862 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:07:52.862 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T18:07:52.862 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T18:07:52.880 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-20T18:07:52.880 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-20T18:07:52.975 INFO:tasks.workunit.client.0.trial195.stderr:pool 'rbd' already exists 2026-02-20T18:07:52.981 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-20T18:07:53.985 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-20T18:07:53.985 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:07:53.986 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:07:53.986 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:07:53.986 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:07:53.986 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:07:53.986 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:07:53.986 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-20T18:07:53.986 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-mon.a.asok log flush 2026-02-20T18:07:54.026 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T18:07:54.031 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-20T18:07:54.032 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T18:07:52.654+0000 7f11d725fd80 0 load: jerasure load: lrc 2026-02-20T18:07:54.032 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_subread_eio_shard_0 td/test-erasure-eio 2026-02-20T18:07:54.032 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:275: TEST_rados_get_subread_eio_shard_0: local dir=td/test-erasure-eio 2026-02-20T18:07:54.032 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:276: TEST_rados_get_subread_eio_shard_0: setup_osds 4 2026-02-20T18:07:54.032 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-20T18:07:54.032 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-20T18:07:54.033 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-20T18:07:54.034 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-20T18:07:54.035 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:07:54.035 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-20T18:07:54.035 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:07:54.035 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:07:54.035 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-20T18:07:54.035 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:07:54.035 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-20T18:07:54.035 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:07:54.035 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:07:54.035 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:07:54.035 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:07:54.035 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-20T18:07:54.035 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-20T18:07:54.035 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:07:54.035 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:07:54.035 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:07:54.035 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:07:54.035 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:07:54.035 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:07:54.036 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:07:54.036 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:07:54.036 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:07:54.036 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:07:54.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:07:54.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:07:54.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:07:54.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:07:54.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:07:54.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:07:54.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:07:54.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:07:54.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:07:54.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:07:54.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:07:54.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:07:54.036 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-20T18:07:54.037 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:07:54.038 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=3fa5a752-7f73-467a-89eb-d016fba80138 2026-02-20T18:07:54.038 INFO:tasks.workunit.client.0.trial195.stdout:add osd0 3fa5a752-7f73-467a-89eb-d016fba80138 2026-02-20T18:07:54.038 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 3fa5a752-7f73-467a-89eb-d016fba80138' 2026-02-20T18:07:54.038 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:07:54.052 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD6ophp/D0CAxAAVyGI4jssI1uNx+CaFktqhg== 2026-02-20T18:07:54.052 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD6ophp/D0CAxAAVyGI4jssI1uNx+CaFktqhg=="}' 2026-02-20T18:07:54.052 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 3fa5a752-7f73-467a-89eb-d016fba80138 -i td/test-erasure-eio/0/new.json 2026-02-20T18:07:54.157 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:07:54.164 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-20T18:07:54.165 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD6ophp/D0CAxAAVyGI4jssI1uNx+CaFktqhg== --osd-uuid 3fa5a752-7f73-467a-89eb-d016fba80138 2026-02-20T18:07:54.183 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:54.180+0000 7f9594e0e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:54.184 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:54.182+0000 7f9594e0e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:54.185 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:54.183+0000 7f9594e0e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:54.186 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:54.183+0000 7f9594e0e900 -1 bdev(0x55fa1cdb4c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-20T18:07:54.186 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:54.183+0000 7f9594e0e900 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-20T18:07:54.609 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-20T18:07:54.609 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:07:54.610 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-20T18:07:54.610 INFO:tasks.workunit.client.0.trial195.stdout:adding osd0 key to auth repository 2026-02-20T18:07:54.610 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:07:54.714 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-20T18:07:54.714 INFO:tasks.workunit.client.0.trial195.stdout:start osd.0 2026-02-20T18:07:54.715 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:07:54.715 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:07:54.715 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:07:54.715 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:07:54.732 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:54.730+0000 7fc98c929900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:54.733 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:54.731+0000 7fc98c929900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:54.734 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:54.732+0000 7fc98c929900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:54.814 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-20T18:07:54.814 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:07:54.814 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-20T18:07:54.814 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:07:54.814 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:07:54.815 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:07:54.815 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:07:54.815 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:07:54.815 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:07:54.816 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T18:07:54.911 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:07:54.951 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:54.949+0000 7fc98c929900 -1 Falling back to public interface 2026-02-20T18:07:55.202 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:55.199+0000 7fc98c929900 -1 osd.0 0 log_to_monitors true 2026-02-20T18:07:55.912 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:07:55.913 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:07:55.913 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:07:55.913 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:07:55.913 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:07:55.913 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T18:07:56.137 INFO:tasks.workunit.client.0.trial195.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/906205159,v1:127.0.0.1:6803/906205159] [v2:127.0.0.1:6804/906205159,v1:127.0.0.1:6805/906205159] exists,up 3fa5a752-7f73-467a-89eb-d016fba80138 2026-02-20T18:07:56.142 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:07:56.142 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:07:56.143 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:07:56.143 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:07:56.143 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-20T18:07:56.143 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:07:56.143 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:07:56.143 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-20T18:07:56.143 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:07:56.143 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-20T18:07:56.143 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:07:56.143 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:07:56.144 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:07:56.144 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:07:56.144 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-20T18:07:56.144 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-20T18:07:56.144 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:07:56.144 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:07:56.144 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:07:56.144 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:07:56.144 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:07:56.144 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:07:56.144 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:07:56.144 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:07:56.145 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:07:56.145 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:07:56.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:07:56.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:07:56.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:07:56.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:07:56.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:07:56.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:07:56.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:07:56.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:07:56.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:07:56.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:07:56.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:07:56.146 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:07:56.146 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-20T18:07:56.146 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:07:56.146 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=87f6988e-a8a8-4e1a-8211-709cfd8a1bf5 2026-02-20T18:07:56.146 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 87f6988e-a8a8-4e1a-8211-709cfd8a1bf5' 2026-02-20T18:07:56.146 INFO:tasks.workunit.client.0.trial195.stdout:add osd1 87f6988e-a8a8-4e1a-8211-709cfd8a1bf5 2026-02-20T18:07:56.147 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:07:56.153 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD8ophpi+IRCRAAXQZO0pzQNP8JddTHwjq/qg== 2026-02-20T18:07:56.153 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD8ophpi+IRCRAAXQZO0pzQNP8JddTHwjq/qg=="}' 2026-02-20T18:07:56.153 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 87f6988e-a8a8-4e1a-8211-709cfd8a1bf5 -i td/test-erasure-eio/1/new.json 2026-02-20T18:07:56.380 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:07:56.388 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-20T18:07:56.389 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD8ophpi+IRCRAAXQZO0pzQNP8JddTHwjq/qg== --osd-uuid 87f6988e-a8a8-4e1a-8211-709cfd8a1bf5 2026-02-20T18:07:56.407 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:56.405+0000 7fcfc3dc9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:56.408 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:56.407+0000 7fcfc3dc9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:56.408 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:56.408+0000 7fcfc3dc9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:56.409 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:56.408+0000 7fcfc3dc9900 -1 bdev(0x563726d20c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-20T18:07:56.409 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:56.408+0000 7fcfc3dc9900 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-20T18:07:56.726 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-20T18:07:56.726 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:07:56.727 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-20T18:07:56.727 INFO:tasks.workunit.client.0.trial195.stdout:adding osd1 key to auth repository 2026-02-20T18:07:56.727 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:07:57.055 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-20T18:07:57.055 INFO:tasks.workunit.client.0.trial195.stdout:start osd.1 2026-02-20T18:07:57.056 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:07:57.056 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:07:57.056 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:07:57.056 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:07:57.073 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:57.072+0000 7f4405aab900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:57.074 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:57.073+0000 7f4405aab900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:57.075 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:57.074+0000 7f4405aab900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:57.215 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:57.214+0000 7f4405aab900 -1 Falling back to public interface 2026-02-20T18:07:57.279 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-20T18:07:57.280 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:07:57.280 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-20T18:07:57.280 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:07:57.280 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:07:57.280 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:07:57.280 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:07:57.280 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:07:57.280 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:07:57.280 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:07:57.505 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:57.504+0000 7f4405aab900 -1 osd.1 0 log_to_monitors true 2026-02-20T18:07:57.506 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:07:58.507 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:07:58.507 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:07:58.508 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:07:58.509 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:07:58.509 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:07:58.509 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:07:58.732 INFO:tasks.workunit.client.0.trial195.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2333512020,v1:127.0.0.1:6811/2333512020] [v2:127.0.0.1:6812/2333512020,v1:127.0.0.1:6813/2333512020] exists,up 87f6988e-a8a8-4e1a-8211-709cfd8a1bf5 2026-02-20T18:07:58.737 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:07:58.737 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:07:58.737 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:07:58.737 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:07:58.737 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-20T18:07:58.737 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:07:58.738 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:07:58.738 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-20T18:07:58.738 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:07:58.738 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-20T18:07:58.738 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:07:58.738 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:07:58.738 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:07:58.738 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:07:58.738 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-20T18:07:58.738 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-20T18:07:58.738 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:07:58.738 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:07:58.739 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:07:58.739 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:07:58.739 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:07:58.739 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:07:58.739 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:07:58.739 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:07:58.739 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:07:58.739 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:07:58.739 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:07:58.739 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:07:58.739 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:07:58.739 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:07:58.740 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:07:58.740 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:07:58.740 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:07:58.740 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:07:58.740 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:07:58.740 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:07:58.740 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:07:58.740 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:07:58.740 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-20T18:07:58.740 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:07:58.740 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=9b10f228-1c6a-4d74-a38f-c85c68962ef0 2026-02-20T18:07:58.740 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 9b10f228-1c6a-4d74-a38f-c85c68962ef0' 2026-02-20T18:07:58.741 INFO:tasks.workunit.client.0.trial195.stdout:add osd2 9b10f228-1c6a-4d74-a38f-c85c68962ef0 2026-02-20T18:07:58.741 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:07:58.749 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD+ophpU6qVLBAAIsW2VLo9BIVPIhPkRlXBoQ== 2026-02-20T18:07:58.749 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD+ophpU6qVLBAAIsW2VLo9BIVPIhPkRlXBoQ=="}' 2026-02-20T18:07:58.750 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 9b10f228-1c6a-4d74-a38f-c85c68962ef0 -i td/test-erasure-eio/2/new.json 2026-02-20T18:07:58.980 INFO:tasks.workunit.client.0.trial195.stdout:2 2026-02-20T18:07:58.987 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-20T18:07:58.988 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD+ophpU6qVLBAAIsW2VLo9BIVPIhPkRlXBoQ== --osd-uuid 9b10f228-1c6a-4d74-a38f-c85c68962ef0 2026-02-20T18:07:59.006 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:59.005+0000 7fe370ea1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:59.007 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:59.006+0000 7fe370ea1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:59.008 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:59.007+0000 7fe370ea1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:59.008 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:59.008+0000 7fe370ea1900 -1 bdev(0x55bad31e6c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-20T18:07:59.009 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:59.008+0000 7fe370ea1900 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-20T18:07:59.335 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-20T18:07:59.335 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:07:59.336 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-20T18:07:59.336 INFO:tasks.workunit.client.0.trial195.stdout:adding osd2 key to auth repository 2026-02-20T18:07:59.337 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:07:59.653 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-20T18:07:59.653 INFO:tasks.workunit.client.0.trial195.stdout:start osd.2 2026-02-20T18:07:59.653 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:07:59.653 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:07:59.653 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:07:59.653 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:07:59.671 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:59.670+0000 7fde9b95b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:59.672 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:59.671+0000 7fde9b95b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:59.673 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:59.672+0000 7fde9b95b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:07:59.812 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:07:59.812+0000 7fde9b95b900 -1 Falling back to public interface 2026-02-20T18:07:59.878 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-20T18:07:59.879 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:07:59.879 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-20T18:07:59.879 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:07:59.879 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:07:59.879 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:07:59.879 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:07:59.879 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:07:59.879 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:07:59.879 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:08:00.103 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:08:00.109 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:00.108+0000 7fde9b95b900 -1 osd.2 0 log_to_monitors true 2026-02-20T18:08:01.104 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:08:01.104 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:08:01.104 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:08:01.105 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:08:01.105 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:08:01.105 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:08:01.330 INFO:tasks.workunit.client.0.trial195.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1220318350,v1:127.0.0.1:6819/1220318350] [v2:127.0.0.1:6820/1220318350,v1:127.0.0.1:6821/1220318350] exists,up 9b10f228-1c6a-4d74-a38f-c85c68962ef0 2026-02-20T18:08:01.330 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:08:01.330 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:08:01.330 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:08:01.330 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:08:01.331 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-20T18:08:01.331 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:08:01.331 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:08:01.331 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-20T18:08:01.331 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:08:01.331 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-20T18:08:01.331 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:08:01.331 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:08:01.331 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:08:01.331 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:08:01.331 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-20T18:08:01.331 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-20T18:08:01.331 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:08:01.331 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:08:01.331 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:08:01.331 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:08:01.331 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:08:01.331 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:08:01.332 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:08:01.332 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:01.332 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:01.332 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:08:01.332 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:08:01.332 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:08:01.332 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:08:01.332 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:08:01.332 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:08:01.332 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:08:01.332 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:08:01.332 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:08:01.332 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:08:01.332 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:08:01.332 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:08:01.332 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:08:01.332 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-20T18:08:01.333 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:08:01.333 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=7029b87f-e9d0-4129-967d-aada1641e0bf 2026-02-20T18:08:01.333 INFO:tasks.workunit.client.0.trial195.stdout:add osd3 7029b87f-e9d0-4129-967d-aada1641e0bf 2026-02-20T18:08:01.333 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 7029b87f-e9d0-4129-967d-aada1641e0bf' 2026-02-20T18:08:01.334 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:08:01.346 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQABo5hplR2QFBAA7fd1RQpz6eghklzTeKphVw== 2026-02-20T18:08:01.346 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQABo5hplR2QFBAA7fd1RQpz6eghklzTeKphVw=="}' 2026-02-20T18:08:01.346 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 7029b87f-e9d0-4129-967d-aada1641e0bf -i td/test-erasure-eio/3/new.json 2026-02-20T18:08:01.574 INFO:tasks.workunit.client.0.trial195.stdout:3 2026-02-20T18:08:01.582 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-20T18:08:01.583 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQABo5hplR2QFBAA7fd1RQpz6eghklzTeKphVw== --osd-uuid 7029b87f-e9d0-4129-967d-aada1641e0bf 2026-02-20T18:08:01.601 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:01.600+0000 7fe951e12900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:01.602 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:01.602+0000 7fe951e12900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:01.603 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:01.603+0000 7fe951e12900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:01.604 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:01.603+0000 7fe951e12900 -1 bdev(0x56177ff2ac00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-20T18:08:01.604 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:01.603+0000 7fe951e12900 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-20T18:08:01.930 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-20T18:08:01.930 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:08:01.931 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-20T18:08:01.931 INFO:tasks.workunit.client.0.trial195.stdout:adding osd3 key to auth repository 2026-02-20T18:08:01.931 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:08:02.251 INFO:tasks.workunit.client.0.trial195.stdout:start osd.3 2026-02-20T18:08:02.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-20T18:08:02.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:08:02.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:08:02.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:08:02.253 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:08:02.268 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:02.267+0000 7f4b4537f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:02.269 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:02.269+0000 7f4b4537f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:02.270 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:02.270+0000 7f4b4537f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:02.404 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:02.402+0000 7f4b4537f900 -1 Falling back to public interface 2026-02-20T18:08:02.476 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-20T18:08:02.476 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:08:02.476 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-20T18:08:02.477 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:08:02.477 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:08:02.477 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:08:02.477 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:08:02.477 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:08:02.478 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:08:02.478 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T18:08:02.697 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:08:02.757 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:02.755+0000 7f4b4537f900 -1 osd.3 0 log_to_monitors true 2026-02-20T18:08:03.698 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:08:03.698 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:08:03.699 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:08:03.699 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:08:03.699 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:08:03.699 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T18:08:03.930 INFO:tasks.workunit.client.0.trial195.stdout:osd.3 up in weight 1 up_from 26 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/650548490,v1:127.0.0.1:6827/650548490] [v2:127.0.0.1:6828/650548490,v1:127.0.0.1:6829/650548490] exists,up 7029b87f-e9d0-4129-967d-aada1641e0bf 2026-02-20T18:08:03.932 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:08:03.932 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:08:03.932 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:08:03.932 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-20T18:08:03.932 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-20T18:08:03.933 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-20T18:08:03.933 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:08:03.933 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:03.933 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:03.933 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.0.asok 2026-02-20T18:08:03.933 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-20T18:08:03.933 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-osd.0.asok log flush 2026-02-20T18:08:03.977 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T18:08:03.981 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-20T18:08:03.983 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T18:07:54.951+0000 7fc98c929900 0 load: jerasure load: lrc 2026-02-20T18:08:03.984 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:278: TEST_rados_get_subread_eio_shard_0: local poolname=pool-jerasure 2026-02-20T18:08:03.984 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:279: TEST_rados_get_subread_eio_shard_0: create_erasure_coded_pool pool-jerasure 2 1 2026-02-20T18:08:03.984 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-20T18:08:03.984 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-20T18:08:03.984 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-20T18:08:03.984 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-20T18:08:03.984 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-20T18:08:03.984 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-20T18:08:03.984 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-20T18:08:04.273 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-20T18:08:04.274 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-20T18:08:04.641 INFO:tasks.workunit.client.0.trial195.stderr:pool 'pool-jerasure' already exists 2026-02-20T18:08:04.648 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-20T18:08:05.649 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-20T18:08:05.650 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:08:05.650 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:08:05.650 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:08:05.650 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:08:05.650 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:08:05.651 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:08:05.651 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:08:05.651 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:08:05.651 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:08:05.737 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:08:05.737 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:08:05.737 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:08:05.737 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:08:05.737 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:08:05.738 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:08:05.964 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:08:05.964 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:08:05.964 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:08:05.964 INFO:tasks.workunit.client.0.trial195.stderr:3' 2026-02-20T18:08:05.964 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:08:05.965 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:08:05.965 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:08:06.027 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803780 2026-02-20T18:08:06.027 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803780 2026-02-20T18:08:06.027 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803780' 2026-02-20T18:08:06.027 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:08:06.027 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:08:06.091 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574852 2026-02-20T18:08:06.092 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574852 2026-02-20T18:08:06.092 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803780 1-55834574852' 2026-02-20T18:08:06.092 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:08:06.092 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:08:06.155 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411332 2026-02-20T18:08:06.155 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411332 2026-02-20T18:08:06.155 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803780 1-55834574852 2-77309411332' 2026-02-20T18:08:06.155 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:08:06.155 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:08:06.218 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149699 2026-02-20T18:08:06.218 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149699 2026-02-20T18:08:06.218 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803780 1-55834574852 2-77309411332 3-111669149699' 2026-02-20T18:08:06.218 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:08:06.218 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803780 2026-02-20T18:08:06.218 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:08:06.219 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:08:06.220 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803780 2026-02-20T18:08:06.220 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:08:06.221 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803780 2026-02-20T18:08:06.221 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803780' 2026-02-20T18:08:06.221 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.0 seq 25769803780 2026-02-20T18:08:06.221 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:08:06.443 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803778 -lt 25769803780 2026-02-20T18:08:06.443 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:08:07.444 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:08:07.444 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:08:07.672 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803781 -lt 25769803780 2026-02-20T18:08:07.673 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:08:07.673 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574852 2026-02-20T18:08:07.673 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:08:07.673 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:08:07.674 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-55834574852 2026-02-20T18:08:07.674 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:08:07.675 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574852 2026-02-20T18:08:07.675 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 55834574852' 2026-02-20T18:08:07.675 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.1 seq 55834574852 2026-02-20T18:08:07.676 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:08:07.902 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574852 -lt 55834574852 2026-02-20T18:08:07.902 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:08:07.903 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411332 2026-02-20T18:08:07.903 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:08:07.904 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:08:07.904 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-77309411332 2026-02-20T18:08:07.904 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:08:07.905 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=77309411332 2026-02-20T18:08:07.906 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 77309411332' 2026-02-20T18:08:07.906 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.2 seq 77309411332 2026-02-20T18:08:07.906 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:08:08.134 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411333 -lt 77309411332 2026-02-20T18:08:08.134 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:08:08.135 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149699 2026-02-20T18:08:08.135 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:08:08.136 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:08:08.137 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149699 2026-02-20T18:08:08.137 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:08:08.138 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149699 2026-02-20T18:08:08.138 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149699' 2026-02-20T18:08:08.138 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.3 seq 111669149699 2026-02-20T18:08:08.139 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:08:08.367 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149699 -lt 111669149699 2026-02-20T18:08:08.367 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:08:08.368 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:08:08.368 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:08:08.682 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:08:08.682 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:08:08.682 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:08:08.682 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:08:08.682 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:08:08.682 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:08:08.683 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:08:08.683 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:08:08.903 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:08:08.903 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:08:08.903 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:08:08.904 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:08:09.217 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:08:09.217 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:08:09.217 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:08:09.217 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:281: TEST_rados_get_subread_eio_shard_0: local shard_id=0 2026-02-20T18:08:09.218 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:282: TEST_rados_get_subread_eio_shard_0: rados_put_get_data eio td/test-erasure-eio 0 2026-02-20T18:08:09.218 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=eio 2026-02-20T18:08:09.218 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-20T18:08:09.218 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-20T18:08:09.218 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-20T18:08:09.218 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=0 2026-02-20T18:08:09.219 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-20T18:08:09.219 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg= 2026-02-20T18:08:09.219 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-20T18:08:09.219 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-eio-85808-0 2026-02-20T18:08:09.219 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-eio-85808-0 2026-02-20T18:08:09.219 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-20T18:08:09.220 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-20T18:08:09.220 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-eio-85808-0 2026-02-20T18:08:09.220 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:08:09.220 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-20T18:08:09.220 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:08:09.220 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-20T18:08:09.221 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:08:09.221 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-20T18:08:09.221 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:08:09.221 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-20T18:08:09.221 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-eio-85808-0 td/test-erasure-eio/ORIGINAL 2026-02-20T18:08:09.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-85808-0 td/test-erasure-eio 0 2026-02-20T18:08:09.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=ec 2026-02-20T18:08:09.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-20T18:08:09.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=data 2026-02-20T18:08:09.238 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-20T18:08:09.238 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=pool-jerasure 2026-02-20T18:08:09.238 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-20T18:08:09.238 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=obj-eio-85808-0 2026-02-20T18:08:09.238 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-20T18:08:09.238 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/test-erasure-eio 2026-02-20T18:08:09.238 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-20T18:08:09.238 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=0 2026-02-20T18:08:09.238 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-20T18:08:09.238 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds pool-jerasure obj-eio-85808-0 2026-02-20T18:08:09.238 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:08:09.238 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-85808-0 2026-02-20T18:08:09.238 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-85808-0 2026-02-20T18:08:09.238 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:08:09.462 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T18:08:09.462 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:08:09.462 INFO:tasks.workunit.client.0.trial195.stderr:2' 2026-02-20T18:08:09.462 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-20T18:08:09.462 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('3' '1' '2') 2026-02-20T18:08:09.462 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-20T18:08:09.462 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=3 2026-02-20T18:08:09.462 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' ec '!=' ec ']' 2026-02-20T18:08:09.463 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/test-erasure-eio/3/type 2026-02-20T18:08:09.463 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-20T18:08:09.463 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-20T18:08:09.463 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-20T18:08:09.464 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-20T18:08:09.464 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-20T18:08:09.464 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-20T18:08:09.464 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-20T18:08:09.464 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-20T18:08:09.464 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-20T18:08:09.464 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-20T18:08:09.465 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:08:09.465 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:09.465 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:09.465 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.3.asok 2026-02-20T18:08:09.465 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.85808/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-20T18:08:09.514 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-20T18:08:09.514 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-20T18:08:09.515 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-20T18:08:09.515 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.3 2026-02-20T18:08:09.515 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-20T18:08:09.515 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-20T18:08:09.516 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:08:09.516 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:09.516 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:09.516 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.3.asok 2026-02-20T18:08:09.516 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-20T18:08:09.517 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-osd.3.asok injectdataerr pool-jerasure obj-eio-85808-0 0 2026-02-20T18:08:09.568 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-85808-0 2026-02-20T18:08:09.569 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-20T18:08:09.569 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-20T18:08:09.569 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-85808-0 2026-02-20T18:08:09.569 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-20T18:08:09.569 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-20T18:08:09.569 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-85808-0 td/test-erasure-eio/COPY 2026-02-20T18:08:09.586 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-20T18:08:09.587 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-20T18:08:09.588 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' '' = recovery ']' 2026-02-20T18:08:09.588 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: expr 0 + 1 2026-02-20T18:08:09.589 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: local another_shard_id=1 2026-02-20T18:08:09.589 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:182: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-85808-0 td/test-erasure-eio 1 2026-02-20T18:08:09.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=ec 2026-02-20T18:08:09.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-20T18:08:09.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=data 2026-02-20T18:08:09.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-20T18:08:09.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=pool-jerasure 2026-02-20T18:08:09.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-20T18:08:09.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=obj-eio-85808-0 2026-02-20T18:08:09.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-20T18:08:09.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/test-erasure-eio 2026-02-20T18:08:09.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-20T18:08:09.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=1 2026-02-20T18:08:09.590 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-20T18:08:09.590 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds pool-jerasure obj-eio-85808-0 2026-02-20T18:08:09.590 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:08:09.590 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-85808-0 2026-02-20T18:08:09.590 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-85808-0 2026-02-20T18:08:09.590 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:08:09.823 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T18:08:09.824 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:08:09.824 INFO:tasks.workunit.client.0.trial195.stderr:2' 2026-02-20T18:08:09.824 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-20T18:08:09.824 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('3' '1' '2') 2026-02-20T18:08:09.824 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-20T18:08:09.825 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=1 2026-02-20T18:08:09.825 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' ec '!=' ec ']' 2026-02-20T18:08:09.825 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/test-erasure-eio/1/type 2026-02-20T18:08:09.825 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-20T18:08:09.825 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-20T18:08:09.825 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-20T18:08:09.826 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-20T18:08:09.826 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-20T18:08:09.826 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-20T18:08:09.826 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-20T18:08:09.826 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-20T18:08:09.826 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-20T18:08:09.826 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-20T18:08:09.827 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:08:09.827 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:09.827 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:09.827 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.1.asok 2026-02-20T18:08:09.827 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.85808/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-20T18:08:09.873 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-20T18:08:09.874 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-20T18:08:09.875 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-20T18:08:09.875 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.1 2026-02-20T18:08:09.875 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-20T18:08:09.875 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-20T18:08:09.876 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:08:09.876 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:09.876 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:09.876 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.1.asok 2026-02-20T18:08:09.876 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-20T18:08:09.876 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-osd.1.asok injectdataerr pool-jerasure obj-eio-85808-0 1 2026-02-20T18:08:09.923 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:183: rados_put_get_data: '[' 0 -eq 1 -a 1 -eq 2 ']' 2026-02-20T18:08:09.923 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:192: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-85808-0 fail 2026-02-20T18:08:09.923 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-20T18:08:09.924 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-20T18:08:09.924 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-85808-0 2026-02-20T18:08:09.924 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-20T18:08:09.924 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-20T18:08:09.924 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-eio-85808-0 td/test-erasure-eio/COPY 2026-02-20T18:08:09.942 INFO:tasks.workunit.client.0.trial195.stderr:error getting pool-jerasure/obj-eio-85808-0: (5) Input/output error 2026-02-20T18:08:09.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-20T18:08:09.944 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:194: rados_put_get_data: rm td/test-erasure-eio/ORIGINAL 2026-02-20T18:08:09.945 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:283: TEST_rados_get_subread_eio_shard_0: delete_erasure_coded_pool pool-jerasure 2026-02-20T18:08:09.945 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-20T18:08:09.945 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-20T18:08:10.215 INFO:tasks.workunit.client.0.trial195.stderr:pool 'pool-jerasure' does not exist 2026-02-20T18:08:10.223 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-20T18:08:10.499 INFO:tasks.workunit.client.0.trial195.stderr:erasure-code-profile myprofile does not exist 2026-02-20T18:08:10.508 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-20T18:08:10.508 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-20T18:08:10.508 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T18:08:10.508 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-20T18:08:10.508 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:08:10.508 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:08:10.509 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:08:10.509 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:08:10.509 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:08:10.617 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:08:10.618 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T18:08:10.619 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T18:08:10.619 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T18:08:10.621 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T18:08:10.621 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T18:08:10.621 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T18:08:10.622 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:08:10.622 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T18:08:10.622 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T18:08:10.622 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:08:10.623 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T18:08:10.624 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T18:08:10.624 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-20T18:08:10.632 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T18:08:10.632 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:10.632 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:10.633 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.85808 2026-02-20T18:08:10.634 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T18:08:10.634 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T18:08:10.634 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-20T18:08:10.634 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-20T18:08:10.634 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-20T18:08:10.634 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-20T18:08:10.634 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-20T18:08:10.634 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T18:08:10.634 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-20T18:08:10.634 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:08:10.635 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:08:10.635 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:08:10.635 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:08:10.635 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:08:10.636 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:08:10.637 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T18:08:10.637 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T18:08:10.638 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T18:08:10.638 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T18:08:10.639 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T18:08:10.639 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T18:08:10.640 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:08:10.640 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T18:08:10.640 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T18:08:10.641 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:08:10.641 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T18:08:10.642 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T18:08:10.642 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-20T18:08:10.643 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T18:08:10.643 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:10.644 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:10.644 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.85808 2026-02-20T18:08:10.644 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T18:08:10.644 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T18:08:10.644 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-20T18:08:10.646 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-20T18:08:10.646 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:10.646 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:10.646 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.85808 2026-02-20T18:08:10.647 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-20T18:08:10.648 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-20T18:08:10.648 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-20T18:08:10.648 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-20T18:08:10.648 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-20T18:08:10.648 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-20T18:08:10.648 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-20T18:08:10.648 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-20T18:08:10.648 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-20T18:08:10.648 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-20T18:08:10.648 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-20T18:08:10.668 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-20T18:08:10.668 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:08:10.668 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:08:10.668 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:08:10.669 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:10.669 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:10.669 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:08:10.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-20T18:08:10.697 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-20T18:08:10.698 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-20T18:08:10.698 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T18:08:10.698 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T18:08:10.698 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-20T18:08:10.698 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-20T18:08:10.699 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T18:08:10.699 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:08:10.699 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:08:10.699 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:08:10.699 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:10.699 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:10.699 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:08:10.699 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T18:08:10.700 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.85808/ceph-mon.a.asok config get fsid 2026-02-20T18:08:10.744 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-20T18:08:10.744 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T18:08:10.744 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T18:08:10.744 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-20T18:08:10.744 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-20T18:08:10.744 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T18:08:10.744 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:08:10.744 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:08:10.745 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:08:10.745 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:10.745 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:10.745 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:08:10.745 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T18:08:10.745 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.85808/ceph-mon.a.asok config get mon_host 2026-02-20T18:08:10.790 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-20T18:08:10.790 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-20T18:08:10.791 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-20T18:08:10.791 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-20T18:08:10.791 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-20T18:08:10.791 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-20T18:08:10.791 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-20T18:08:10.894 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-20T18:08:10.895 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:08:10.895 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:08:10.895 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:08:10.895 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:10.895 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:10.895 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:08:10.895 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T18:08:10.896 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T18:08:10.913 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-20T18:08:10.913 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-20T18:08:11.008 INFO:tasks.workunit.client.0.trial195.stderr:pool 'rbd' already exists 2026-02-20T18:08:11.015 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-20T18:08:12.016 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-20T18:08:12.016 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:08:12.016 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:08:12.016 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:08:12.016 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:12.016 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:12.017 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:08:12.017 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-20T18:08:12.017 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-mon.a.asok log flush 2026-02-20T18:08:12.060 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T18:08:12.065 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-20T18:08:12.066 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T18:08:10.689+0000 7f22bf012d80 0 load: jerasure load: lrc 2026-02-20T18:08:12.067 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_subread_eio_shard_1 td/test-erasure-eio 2026-02-20T18:08:12.067 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:287: TEST_rados_get_subread_eio_shard_1: local dir=td/test-erasure-eio 2026-02-20T18:08:12.068 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:288: TEST_rados_get_subread_eio_shard_1: setup_osds 4 2026-02-20T18:08:12.068 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-20T18:08:12.068 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-20T18:08:12.068 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-20T18:08:12.068 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-20T18:08:12.072 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:08:12.072 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-20T18:08:12.072 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:08:12.072 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:08:12.072 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-20T18:08:12.072 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:08:12.073 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-20T18:08:12.073 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:08:12.073 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:08:12.073 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:08:12.073 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:08:12.073 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-20T18:08:12.073 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-20T18:08:12.073 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:08:12.073 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:08:12.073 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:08:12.074 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:08:12.074 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:08:12.074 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:08:12.074 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:08:12.074 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:12.074 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:12.074 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:08:12.074 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:08:12.074 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:08:12.074 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:08:12.074 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:08:12.074 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:08:12.075 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:08:12.075 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:08:12.075 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:08:12.075 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:08:12.075 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:08:12.075 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:08:12.075 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:08:12.075 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-20T18:08:12.075 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:08:12.075 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=cb6171e9-2756-45e9-80ff-f440a835697d 2026-02-20T18:08:12.075 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 cb6171e9-2756-45e9-80ff-f440a835697d' 2026-02-20T18:08:12.076 INFO:tasks.workunit.client.0.trial195.stdout:add osd0 cb6171e9-2756-45e9-80ff-f440a835697d 2026-02-20T18:08:12.076 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:08:12.084 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAMo5hpHgXwBBAArxBQUo1bkVZMbAAC/I3mtQ== 2026-02-20T18:08:12.084 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAMo5hpHgXwBBAArxBQUo1bkVZMbAAC/I3mtQ=="}' 2026-02-20T18:08:12.084 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new cb6171e9-2756-45e9-80ff-f440a835697d -i td/test-erasure-eio/0/new.json 2026-02-20T18:08:12.190 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:08:12.197 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-20T18:08:12.198 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAMo5hpHgXwBBAArxBQUo1bkVZMbAAC/I3mtQ== --osd-uuid cb6171e9-2756-45e9-80ff-f440a835697d 2026-02-20T18:08:12.216 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:12.215+0000 7fb01cd57900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:12.218 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:12.217+0000 7fb01cd57900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:12.219 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:12.218+0000 7fb01cd57900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:12.219 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:12.218+0000 7fb01cd57900 -1 bdev(0x558228cc4c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-20T18:08:12.219 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:12.218+0000 7fb01cd57900 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-20T18:08:12.658 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-20T18:08:12.658 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:08:12.659 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-20T18:08:12.659 INFO:tasks.workunit.client.0.trial195.stdout:adding osd0 key to auth repository 2026-02-20T18:08:12.659 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:08:12.765 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-20T18:08:12.765 INFO:tasks.workunit.client.0.trial195.stdout:start osd.0 2026-02-20T18:08:12.766 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:08:12.766 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:08:12.766 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:08:12.766 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:08:12.782 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:12.781+0000 7f9f91157900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:12.783 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:12.782+0000 7f9f91157900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:12.784 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:12.783+0000 7f9f91157900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:12.863 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-20T18:08:12.863 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:08:12.864 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-20T18:08:12.864 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:08:12.864 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:08:12.864 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:08:12.864 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:08:12.864 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:08:12.865 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:08:12.865 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T18:08:12.962 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:08:13.006 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:13.005+0000 7f9f91157900 -1 Falling back to public interface 2026-02-20T18:08:13.258 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:13.257+0000 7f9f91157900 -1 osd.0 0 log_to_monitors true 2026-02-20T18:08:13.963 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:08:13.963 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:08:13.963 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:08:13.964 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:08:13.964 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:08:13.964 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T18:08:14.188 INFO:tasks.workunit.client.0.trial195.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/33439336,v1:127.0.0.1:6803/33439336] [v2:127.0.0.1:6804/33439336,v1:127.0.0.1:6805/33439336] exists,up cb6171e9-2756-45e9-80ff-f440a835697d 2026-02-20T18:08:14.190 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:08:14.190 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:08:14.190 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:08:14.190 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:08:14.190 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-20T18:08:14.190 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:08:14.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:08:14.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-20T18:08:14.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:08:14.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-20T18:08:14.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:08:14.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:08:14.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:08:14.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:08:14.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-20T18:08:14.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-20T18:08:14.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:08:14.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:08:14.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:08:14.191 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:08:14.191 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:08:14.191 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:08:14.191 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:08:14.191 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:14.191 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:14.191 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:08:14.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:08:14.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:08:14.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:08:14.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:08:14.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:08:14.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:08:14.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:08:14.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:08:14.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:08:14.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:08:14.192 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:08:14.192 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:08:14.192 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-20T18:08:14.192 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:08:14.192 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=025b2dda-364d-4b75-8905-011100a7606d 2026-02-20T18:08:14.192 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 025b2dda-364d-4b75-8905-011100a7606d' 2026-02-20T18:08:14.192 INFO:tasks.workunit.client.0.trial195.stdout:add osd1 025b2dda-364d-4b75-8905-011100a7606d 2026-02-20T18:08:14.192 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:08:14.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAOo5hpwREcDBAAO6NsGJqONUvN3kV460DR0A== 2026-02-20T18:08:14.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAOo5hpwREcDBAAO6NsGJqONUvN3kV460DR0A=="}' 2026-02-20T18:08:14.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 025b2dda-364d-4b75-8905-011100a7606d -i td/test-erasure-eio/1/new.json 2026-02-20T18:08:14.436 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:08:14.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-20T18:08:14.446 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAOo5hpwREcDBAAO6NsGJqONUvN3kV460DR0A== --osd-uuid 025b2dda-364d-4b75-8905-011100a7606d 2026-02-20T18:08:14.464 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:14.463+0000 7fcb22a0c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:14.465 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:14.465+0000 7fcb22a0c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:14.466 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:14.466+0000 7fcb22a0c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:14.467 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:14.466+0000 7fcb22a0c900 -1 bdev(0x5648a898ec00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-20T18:08:14.467 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:14.466+0000 7fcb22a0c900 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-20T18:08:14.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-20T18:08:14.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:08:14.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-20T18:08:14.798 INFO:tasks.workunit.client.0.trial195.stdout:adding osd1 key to auth repository 2026-02-20T18:08:14.799 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:08:15.121 INFO:tasks.workunit.client.0.trial195.stdout:start osd.1 2026-02-20T18:08:15.122 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-20T18:08:15.122 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:08:15.122 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:08:15.122 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:08:15.122 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:08:15.139 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:15.138+0000 7f75bc014900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:15.140 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:15.139+0000 7f75bc014900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:15.141 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:15.140+0000 7f75bc014900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:15.272 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:15.270+0000 7f75bc014900 -1 Falling back to public interface 2026-02-20T18:08:15.351 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-20T18:08:15.351 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:08:15.351 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-20T18:08:15.351 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:08:15.351 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:08:15.351 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:08:15.352 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:08:15.352 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:08:15.352 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:08:15.353 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:08:15.574 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:08:15.636 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:15.635+0000 7f75bc014900 -1 osd.1 0 log_to_monitors true 2026-02-20T18:08:16.576 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:08:16.576 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:08:16.576 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:08:16.576 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:08:16.576 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:08:16.576 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:08:16.800 INFO:tasks.workunit.client.0.trial195.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2690702902,v1:127.0.0.1:6811/2690702902] [v2:127.0.0.1:6812/2690702902,v1:127.0.0.1:6813/2690702902] exists,up 025b2dda-364d-4b75-8905-011100a7606d 2026-02-20T18:08:16.804 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:08:16.804 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:08:16.804 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:08:16.804 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:08:16.804 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-20T18:08:16.804 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:08:16.804 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:08:16.804 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-20T18:08:16.804 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:08:16.804 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-20T18:08:16.805 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:08:16.805 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:08:16.805 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:08:16.805 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:08:16.805 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-20T18:08:16.805 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-20T18:08:16.805 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:08:16.805 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:08:16.805 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:08:16.805 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:08:16.805 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:08:16.806 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:08:16.806 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:08:16.806 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:16.806 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:16.806 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:08:16.806 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:08:16.806 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:08:16.806 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:08:16.806 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:08:16.806 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:08:16.806 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:08:16.806 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:08:16.807 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:08:16.807 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:08:16.807 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:08:16.807 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:08:16.807 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:08:16.807 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-20T18:08:16.807 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:08:16.808 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=147ff30e-76e3-4475-8841-0692a31328af 2026-02-20T18:08:16.808 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 147ff30e-76e3-4475-8841-0692a31328af' 2026-02-20T18:08:16.808 INFO:tasks.workunit.client.0.trial195.stdout:add osd2 147ff30e-76e3-4475-8841-0692a31328af 2026-02-20T18:08:16.809 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:08:16.816 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAQo5hp67GUMBAAOuMD+S7lBUWXlBjSb/g60A== 2026-02-20T18:08:16.816 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAQo5hp67GUMBAAOuMD+S7lBUWXlBjSb/g60A=="}' 2026-02-20T18:08:16.816 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 147ff30e-76e3-4475-8841-0692a31328af -i td/test-erasure-eio/2/new.json 2026-02-20T18:08:17.041 INFO:tasks.workunit.client.0.trial195.stdout:2 2026-02-20T18:08:17.048 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-20T18:08:17.049 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAQo5hp67GUMBAAOuMD+S7lBUWXlBjSb/g60A== --osd-uuid 147ff30e-76e3-4475-8841-0692a31328af 2026-02-20T18:08:17.067 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:17.066+0000 7f0d1ea25900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:17.068 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:17.067+0000 7f0d1ea25900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:17.069 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:17.068+0000 7f0d1ea25900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:17.070 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:17.069+0000 7f0d1ea25900 -1 bdev(0x560dcfb74c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-20T18:08:17.070 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:17.069+0000 7f0d1ea25900 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-20T18:08:17.380 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-20T18:08:17.380 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:08:17.381 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-20T18:08:17.381 INFO:tasks.workunit.client.0.trial195.stdout:adding osd2 key to auth repository 2026-02-20T18:08:17.381 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:08:17.693 INFO:tasks.workunit.client.0.trial195.stdout:start osd.2 2026-02-20T18:08:17.694 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-20T18:08:17.694 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:08:17.694 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:08:17.694 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:08:17.694 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:08:17.711 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:17.710+0000 7f6d13410900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:17.712 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:17.711+0000 7f6d13410900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:17.713 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:17.712+0000 7f6d13410900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:17.829 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:17.827+0000 7f6d13410900 -1 Falling back to public interface 2026-02-20T18:08:17.919 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-20T18:08:17.920 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:08:17.920 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-20T18:08:17.920 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:08:17.920 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:08:17.920 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:08:17.920 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:08:17.921 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:08:17.921 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:08:17.921 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:08:18.147 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:08:18.175 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:18.174+0000 7f6d13410900 -1 osd.2 0 log_to_monitors true 2026-02-20T18:08:19.148 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:08:19.149 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:08:19.149 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:08:19.150 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:08:19.150 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:08:19.150 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:08:19.369 INFO:tasks.workunit.client.0.trial195.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3005755224,v1:127.0.0.1:6819/3005755224] [v2:127.0.0.1:6820/3005755224,v1:127.0.0.1:6821/3005755224] exists,up 147ff30e-76e3-4475-8841-0692a31328af 2026-02-20T18:08:19.373 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:08:19.373 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:08:19.374 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:08:19.374 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:08:19.374 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-20T18:08:19.374 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:08:19.374 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:08:19.374 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-20T18:08:19.374 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:08:19.374 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-20T18:08:19.374 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:08:19.374 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:08:19.374 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:08:19.375 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:08:19.375 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-20T18:08:19.375 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-20T18:08:19.375 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:08:19.375 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:08:19.375 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:08:19.375 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:08:19.375 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:08:19.375 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:08:19.375 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:08:19.375 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:19.375 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:19.376 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:08:19.376 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:08:19.376 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:08:19.376 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:08:19.376 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:08:19.376 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:08:19.376 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:08:19.376 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:08:19.376 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:08:19.376 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:08:19.377 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:08:19.377 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:08:19.377 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:08:19.377 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-20T18:08:19.377 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:08:19.377 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=258ee290-93f1-4859-b51a-0a1dd6e0bc03 2026-02-20T18:08:19.377 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 258ee290-93f1-4859-b51a-0a1dd6e0bc03' 2026-02-20T18:08:19.378 INFO:tasks.workunit.client.0.trial195.stdout:add osd3 258ee290-93f1-4859-b51a-0a1dd6e0bc03 2026-02-20T18:08:19.378 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:08:19.385 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQATo5hplxzhFhAADSc53sAw+KQnnxnmA96cpg== 2026-02-20T18:08:19.385 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQATo5hplxzhFhAADSc53sAw+KQnnxnmA96cpg=="}' 2026-02-20T18:08:19.385 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 258ee290-93f1-4859-b51a-0a1dd6e0bc03 -i td/test-erasure-eio/3/new.json 2026-02-20T18:08:19.613 INFO:tasks.workunit.client.0.trial195.stdout:3 2026-02-20T18:08:19.622 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-20T18:08:19.624 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQATo5hplxzhFhAADSc53sAw+KQnnxnmA96cpg== --osd-uuid 258ee290-93f1-4859-b51a-0a1dd6e0bc03 2026-02-20T18:08:19.642 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:19.641+0000 7f0adb4a1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:19.643 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:19.643+0000 7f0adb4a1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:19.644 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:19.644+0000 7f0adb4a1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:19.645 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:19.644+0000 7f0adb4a1900 -1 bdev(0x55c82a97cc00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-20T18:08:19.645 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:19.644+0000 7f0adb4a1900 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-20T18:08:19.978 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-20T18:08:19.978 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:08:19.979 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-20T18:08:19.979 INFO:tasks.workunit.client.0.trial195.stdout:adding osd3 key to auth repository 2026-02-20T18:08:19.979 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:08:20.294 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-20T18:08:20.295 INFO:tasks.workunit.client.0.trial195.stdout:start osd.3 2026-02-20T18:08:20.295 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:08:20.295 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:08:20.295 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:08:20.295 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:08:20.311 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:20.311+0000 7fc2a8ae2900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:20.313 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:20.312+0000 7fc2a8ae2900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:20.314 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:20.313+0000 7fc2a8ae2900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:20.440 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:20.439+0000 7fc2a8ae2900 -1 Falling back to public interface 2026-02-20T18:08:20.521 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-20T18:08:20.521 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:08:20.521 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-20T18:08:20.522 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:08:20.522 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:08:20.522 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:08:20.522 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:08:20.523 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:08:20.523 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:08:20.524 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T18:08:20.736 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:20.736+0000 7fc2a8ae2900 -1 osd.3 0 log_to_monitors true 2026-02-20T18:08:20.748 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:08:21.749 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:08:21.749 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:08:21.749 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:08:21.750 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:08:21.750 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:08:21.750 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T18:08:21.976 INFO:tasks.workunit.client.0.trial195.stdout:osd.3 up in weight 1 up_from 26 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/919346278,v1:127.0.0.1:6827/919346278] [v2:127.0.0.1:6828/919346278,v1:127.0.0.1:6829/919346278] exists,up 258ee290-93f1-4859-b51a-0a1dd6e0bc03 2026-02-20T18:08:21.977 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:08:21.977 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:08:21.977 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:08:21.978 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-20T18:08:21.978 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-20T18:08:21.978 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-20T18:08:21.978 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:08:21.978 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:21.978 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:21.979 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.0.asok 2026-02-20T18:08:21.979 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-20T18:08:21.979 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-osd.0.asok log flush 2026-02-20T18:08:22.025 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T18:08:22.029 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-20T18:08:22.031 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T18:08:13.007+0000 7f9f91157900 0 load: jerasure load: lrc 2026-02-20T18:08:22.032 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:290: TEST_rados_get_subread_eio_shard_1: local poolname=pool-jerasure 2026-02-20T18:08:22.032 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:291: TEST_rados_get_subread_eio_shard_1: create_erasure_coded_pool pool-jerasure 2 1 2026-02-20T18:08:22.032 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-20T18:08:22.032 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-20T18:08:22.033 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-20T18:08:22.033 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-20T18:08:22.033 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-20T18:08:22.033 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-20T18:08:22.033 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-20T18:08:22.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-20T18:08:22.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-20T18:08:22.691 INFO:tasks.workunit.client.0.trial195.stderr:pool 'pool-jerasure' already exists 2026-02-20T18:08:22.699 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-20T18:08:23.701 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-20T18:08:23.701 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:08:23.702 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:08:23.702 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:08:23.702 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:08:23.703 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:08:23.703 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:08:23.703 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:08:23.703 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:08:23.703 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:08:23.791 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:08:23.791 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:08:23.791 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:08:23.791 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:08:23.791 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:08:23.791 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:08:24.015 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:08:24.015 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:08:24.015 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:08:24.015 INFO:tasks.workunit.client.0.trial195.stderr:3' 2026-02-20T18:08:24.015 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:08:24.015 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:08:24.015 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:08:24.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803781 2026-02-20T18:08:24.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803781 2026-02-20T18:08:24.080 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781' 2026-02-20T18:08:24.080 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:08:24.080 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:08:24.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607556 2026-02-20T18:08:24.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607556 2026-02-20T18:08:24.145 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781 1-51539607556' 2026-02-20T18:08:24.146 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:08:24.146 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:08:24.210 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411331 2026-02-20T18:08:24.211 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411331 2026-02-20T18:08:24.211 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781 1-51539607556 2-77309411331' 2026-02-20T18:08:24.211 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:08:24.211 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:08:24.275 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149700 2026-02-20T18:08:24.275 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149700 2026-02-20T18:08:24.275 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781 1-51539607556 2-77309411331 3-111669149700' 2026-02-20T18:08:24.275 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:08:24.275 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803781 2026-02-20T18:08:24.275 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:08:24.276 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:08:24.276 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803781 2026-02-20T18:08:24.277 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:08:24.278 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803781 2026-02-20T18:08:24.278 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-20T18:08:24.278 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.0 seq 25769803781 2026-02-20T18:08:24.278 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:08:24.503 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803779 -lt 25769803781 2026-02-20T18:08:24.503 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:08:25.504 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:08:25.505 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:08:25.732 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803782 -lt 25769803781 2026-02-20T18:08:25.733 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:08:25.733 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607556 2026-02-20T18:08:25.733 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:08:25.733 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:08:25.734 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-51539607556 2026-02-20T18:08:25.734 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:08:25.735 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=51539607556 2026-02-20T18:08:25.735 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 51539607556' 2026-02-20T18:08:25.736 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.1 seq 51539607556 2026-02-20T18:08:25.736 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:08:25.962 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 51539607556 -lt 51539607556 2026-02-20T18:08:25.962 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:08:25.962 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411331 2026-02-20T18:08:25.963 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:08:25.963 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:08:25.964 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-77309411331 2026-02-20T18:08:25.964 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:08:25.965 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=77309411331 2026-02-20T18:08:25.965 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.2 seq 77309411331 2026-02-20T18:08:25.965 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 77309411331' 2026-02-20T18:08:25.966 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:08:26.195 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411332 -lt 77309411331 2026-02-20T18:08:26.195 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:08:26.195 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149700 2026-02-20T18:08:26.195 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:08:26.196 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:08:26.197 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149700 2026-02-20T18:08:26.197 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:08:26.198 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149700 2026-02-20T18:08:26.198 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.3 seq 111669149700 2026-02-20T18:08:26.198 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149700' 2026-02-20T18:08:26.198 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:08:26.422 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149700 -lt 111669149700 2026-02-20T18:08:26.423 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:08:26.423 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:08:26.423 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:08:26.735 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:08:26.735 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:08:26.735 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:08:26.735 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:08:26.735 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:08:26.735 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:08:26.735 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:08:26.736 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:08:26.958 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:08:26.958 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:08:26.958 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:08:26.959 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:08:27.270 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:08:27.270 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:08:27.270 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:08:27.270 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:293: TEST_rados_get_subread_eio_shard_1: local shard_id=1 2026-02-20T18:08:27.270 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:294: TEST_rados_get_subread_eio_shard_1: rados_put_get_data eio td/test-erasure-eio 1 2026-02-20T18:08:27.270 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=eio 2026-02-20T18:08:27.270 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-20T18:08:27.270 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-20T18:08:27.270 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-20T18:08:27.270 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=1 2026-02-20T18:08:27.270 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-20T18:08:27.270 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg= 2026-02-20T18:08:27.270 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-20T18:08:27.270 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-eio-85808-1 2026-02-20T18:08:27.270 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-eio-85808-1 2026-02-20T18:08:27.271 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-20T18:08:27.271 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-20T18:08:27.271 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-eio-85808-1 2026-02-20T18:08:27.271 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:08:27.271 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-20T18:08:27.271 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:08:27.271 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-20T18:08:27.271 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:08:27.271 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-20T18:08:27.271 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:08:27.271 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-20T18:08:27.271 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-eio-85808-1 td/test-erasure-eio/ORIGINAL 2026-02-20T18:08:27.290 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-85808-1 td/test-erasure-eio 1 2026-02-20T18:08:27.290 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=ec 2026-02-20T18:08:27.290 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-20T18:08:27.290 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=data 2026-02-20T18:08:27.290 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-20T18:08:27.290 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=pool-jerasure 2026-02-20T18:08:27.290 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-20T18:08:27.290 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=obj-eio-85808-1 2026-02-20T18:08:27.290 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-20T18:08:27.290 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/test-erasure-eio 2026-02-20T18:08:27.290 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-20T18:08:27.290 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=1 2026-02-20T18:08:27.290 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-20T18:08:27.290 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds pool-jerasure obj-eio-85808-1 2026-02-20T18:08:27.290 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:08:27.290 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-85808-1 2026-02-20T18:08:27.291 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-85808-1 2026-02-20T18:08:27.291 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:08:27.516 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T18:08:27.516 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:08:27.517 INFO:tasks.workunit.client.0.trial195.stderr:2' 2026-02-20T18:08:27.517 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-20T18:08:27.517 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('3' '1' '2') 2026-02-20T18:08:27.517 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-20T18:08:27.517 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=1 2026-02-20T18:08:27.517 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' ec '!=' ec ']' 2026-02-20T18:08:27.517 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/test-erasure-eio/1/type 2026-02-20T18:08:27.519 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-20T18:08:27.519 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-20T18:08:27.519 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-20T18:08:27.519 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-20T18:08:27.520 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-20T18:08:27.520 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-20T18:08:27.520 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-20T18:08:27.520 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-20T18:08:27.520 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-20T18:08:27.520 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-20T18:08:27.520 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:08:27.520 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:27.520 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:27.520 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.1.asok 2026-02-20T18:08:27.520 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.85808/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-20T18:08:27.568 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-20T18:08:27.568 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-20T18:08:27.570 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-20T18:08:27.570 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.1 2026-02-20T18:08:27.570 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-20T18:08:27.570 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-20T18:08:27.570 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:08:27.570 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:27.570 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:27.570 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.1.asok 2026-02-20T18:08:27.570 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-20T18:08:27.570 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-osd.1.asok injectdataerr pool-jerasure obj-eio-85808-1 1 2026-02-20T18:08:27.618 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-85808-1 2026-02-20T18:08:27.618 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-20T18:08:27.618 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-20T18:08:27.618 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-85808-1 2026-02-20T18:08:27.618 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-20T18:08:27.618 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-20T18:08:27.619 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-85808-1 td/test-erasure-eio/COPY 2026-02-20T18:08:27.637 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-20T18:08:27.638 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-20T18:08:27.639 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' '' = recovery ']' 2026-02-20T18:08:27.639 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: expr 1 + 1 2026-02-20T18:08:27.642 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: local another_shard_id=2 2026-02-20T18:08:27.642 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:182: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-85808-1 td/test-erasure-eio 2 2026-02-20T18:08:27.642 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=ec 2026-02-20T18:08:27.642 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-20T18:08:27.642 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=data 2026-02-20T18:08:27.642 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-20T18:08:27.642 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=pool-jerasure 2026-02-20T18:08:27.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-20T18:08:27.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=obj-eio-85808-1 2026-02-20T18:08:27.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-20T18:08:27.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/test-erasure-eio 2026-02-20T18:08:27.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-20T18:08:27.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=2 2026-02-20T18:08:27.643 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-20T18:08:27.643 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds pool-jerasure obj-eio-85808-1 2026-02-20T18:08:27.643 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:08:27.643 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-85808-1 2026-02-20T18:08:27.643 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-85808-1 2026-02-20T18:08:27.643 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:08:27.866 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T18:08:27.866 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:08:27.866 INFO:tasks.workunit.client.0.trial195.stderr:2' 2026-02-20T18:08:27.866 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-20T18:08:27.866 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('3' '1' '2') 2026-02-20T18:08:27.866 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-20T18:08:27.866 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=2 2026-02-20T18:08:27.867 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' ec '!=' ec ']' 2026-02-20T18:08:27.867 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/test-erasure-eio/2/type 2026-02-20T18:08:27.867 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-20T18:08:27.868 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 2 bluestore_debug_inject_read_err true 2026-02-20T18:08:27.868 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-20T18:08:27.868 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=2 2026-02-20T18:08:27.868 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-20T18:08:27.868 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-20T18:08:27.869 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-20T18:08:27.869 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.2 2026-02-20T18:08:27.869 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.2 2026-02-20T18:08:27.869 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.2 ']' 2026-02-20T18:08:27.869 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:08:27.869 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:27.869 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:27.869 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.2.asok 2026-02-20T18:08:27.869 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.85808/ceph-osd.2.asok config set bluestore_debug_inject_read_err true 2026-02-20T18:08:27.917 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-20T18:08:27.917 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-20T18:08:27.918 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-20T18:08:27.918 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.2 2026-02-20T18:08:27.918 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.2 2026-02-20T18:08:27.918 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.2 ']' 2026-02-20T18:08:27.919 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:08:27.919 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:27.919 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:27.919 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.2.asok 2026-02-20T18:08:27.919 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-20T18:08:27.919 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-osd.2.asok injectdataerr pool-jerasure obj-eio-85808-1 2 2026-02-20T18:08:27.967 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:183: rados_put_get_data: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-20T18:08:27.967 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:188: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-85808-1 2026-02-20T18:08:27.967 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-20T18:08:27.967 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-20T18:08:27.967 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-85808-1 2026-02-20T18:08:27.967 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-20T18:08:27.967 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-20T18:08:27.967 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-85808-1 td/test-erasure-eio/COPY 2026-02-20T18:08:27.986 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-20T18:08:27.987 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-20T18:08:27.987 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:194: rados_put_get_data: rm td/test-erasure-eio/ORIGINAL 2026-02-20T18:08:27.988 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:295: TEST_rados_get_subread_eio_shard_1: delete_erasure_coded_pool pool-jerasure 2026-02-20T18:08:27.988 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-20T18:08:27.988 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-20T18:08:28.257 INFO:tasks.workunit.client.0.trial195.stderr:pool 'pool-jerasure' does not exist 2026-02-20T18:08:28.265 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-20T18:08:28.546 INFO:tasks.workunit.client.0.trial195.stderr:erasure-code-profile myprofile does not exist 2026-02-20T18:08:28.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-20T18:08:28.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-20T18:08:28.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T18:08:28.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-20T18:08:28.555 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:08:28.555 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:08:28.555 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:08:28.555 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:08:28.555 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:08:28.664 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:08:28.665 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T18:08:28.666 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T18:08:28.666 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T18:08:28.668 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T18:08:28.668 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T18:08:28.668 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T18:08:28.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:08:28.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T18:08:28.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T18:08:28.670 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:08:28.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T18:08:28.671 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T18:08:28.671 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-20T18:08:28.680 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T18:08:28.680 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:28.680 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:28.681 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.85808 2026-02-20T18:08:28.682 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T18:08:28.682 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T18:08:28.683 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-20T18:08:28.683 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-20T18:08:28.683 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-20T18:08:28.683 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-20T18:08:28.683 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-20T18:08:28.683 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T18:08:28.683 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-20T18:08:28.683 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:08:28.683 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:08:28.683 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:08:28.683 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:08:28.683 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:08:28.684 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:08:28.685 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T18:08:28.685 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T18:08:28.685 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T18:08:28.685 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T18:08:28.685 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T18:08:28.686 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T18:08:28.687 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:08:28.687 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T18:08:28.688 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T18:08:28.688 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:08:28.688 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T18:08:28.689 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T18:08:28.689 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-20T18:08:28.690 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T18:08:28.690 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:28.691 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:28.691 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.85808 2026-02-20T18:08:28.691 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T18:08:28.692 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T18:08:28.692 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-20T18:08:28.693 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-20T18:08:28.693 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:28.693 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:28.693 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.85808 2026-02-20T18:08:28.695 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-20T18:08:28.696 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-20T18:08:28.696 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-20T18:08:28.696 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-20T18:08:28.696 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-20T18:08:28.696 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-20T18:08:28.696 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-20T18:08:28.696 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-20T18:08:28.696 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-20T18:08:28.696 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-20T18:08:28.696 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-20T18:08:28.717 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-20T18:08:28.717 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:08:28.718 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:08:28.718 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:08:28.718 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:28.718 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:28.718 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:08:28.718 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-20T18:08:28.746 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-20T18:08:28.747 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-20T18:08:28.747 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T18:08:28.748 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T18:08:28.748 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-20T18:08:28.748 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-20T18:08:28.748 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T18:08:28.748 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:08:28.748 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:08:28.749 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:08:28.749 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:28.749 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:28.749 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:08:28.749 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T18:08:28.749 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.85808/ceph-mon.a.asok config get fsid 2026-02-20T18:08:28.797 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-20T18:08:28.798 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T18:08:28.798 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T18:08:28.798 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-20T18:08:28.798 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-20T18:08:28.798 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T18:08:28.799 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:08:28.799 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:08:28.799 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:08:28.799 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:28.799 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:28.799 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:08:28.800 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T18:08:28.800 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.85808/ceph-mon.a.asok config get mon_host 2026-02-20T18:08:28.841 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-20T18:08:28.841 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-20T18:08:28.842 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-20T18:08:28.842 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-20T18:08:28.842 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-20T18:08:28.842 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-20T18:08:28.842 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-20T18:08:28.941 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-20T18:08:28.941 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:08:28.941 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:08:28.941 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:08:28.941 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:28.941 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:28.941 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:08:28.942 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T18:08:28.942 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T18:08:28.958 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-20T18:08:28.958 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-20T18:08:29.056 INFO:tasks.workunit.client.0.trial195.stderr:pool 'rbd' already exists 2026-02-20T18:08:29.062 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-20T18:08:30.065 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-20T18:08:30.065 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:08:30.065 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:08:30.066 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:08:30.066 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:30.066 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:30.066 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:08:30.066 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-20T18:08:30.066 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-mon.a.asok log flush 2026-02-20T18:08:30.107 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T18:08:30.111 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-20T18:08:30.112 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T18:08:28.737+0000 7f2aaf3cad80 0 load: jerasure load: lrc 2026-02-20T18:08:30.113 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_subread_missing td/test-erasure-eio 2026-02-20T18:08:30.114 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:302: TEST_rados_get_subread_missing: local dir=td/test-erasure-eio 2026-02-20T18:08:30.114 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:303: TEST_rados_get_subread_missing: setup_osds 4 2026-02-20T18:08:30.114 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-20T18:08:30.114 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-20T18:08:30.114 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-20T18:08:30.114 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-20T18:08:30.119 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:08:30.119 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-20T18:08:30.119 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:08:30.119 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:08:30.119 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-20T18:08:30.119 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:08:30.119 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-20T18:08:30.119 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:08:30.119 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:08:30.120 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:08:30.120 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:08:30.120 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-20T18:08:30.120 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-20T18:08:30.120 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:08:30.120 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:08:30.120 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:08:30.120 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:08:30.120 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:08:30.120 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:08:30.120 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:08:30.121 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:30.121 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:30.121 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:08:30.121 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:08:30.121 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:08:30.121 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:08:30.121 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:08:30.121 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:08:30.121 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:08:30.121 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:08:30.121 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:08:30.121 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:08:30.121 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:08:30.122 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:08:30.122 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:08:30.122 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-20T18:08:30.122 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:08:30.122 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8187a2a4-6542-490d-b430-721555c01efb 2026-02-20T18:08:30.123 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 8187a2a4-6542-490d-b430-721555c01efb' 2026-02-20T18:08:30.123 INFO:tasks.workunit.client.0.trial195.stdout:add osd0 8187a2a4-6542-490d-b430-721555c01efb 2026-02-20T18:08:30.123 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:08:30.133 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAeo5hp7+3ZBxAAYMMAQpFgtMMpd4sv4Bh0eQ== 2026-02-20T18:08:30.133 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAeo5hp7+3ZBxAAYMMAQpFgtMMpd4sv4Bh0eQ=="}' 2026-02-20T18:08:30.134 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8187a2a4-6542-490d-b430-721555c01efb -i td/test-erasure-eio/0/new.json 2026-02-20T18:08:30.239 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:08:30.245 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-20T18:08:30.246 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAeo5hp7+3ZBxAAYMMAQpFgtMMpd4sv4Bh0eQ== --osd-uuid 8187a2a4-6542-490d-b430-721555c01efb 2026-02-20T18:08:30.264 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:30.262+0000 7f97af697900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:30.267 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:30.265+0000 7f97af697900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:30.268 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:30.266+0000 7f97af697900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:30.268 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:30.266+0000 7f97af697900 -1 bdev(0x55a802cf6c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-20T18:08:30.268 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:30.266+0000 7f97af697900 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-20T18:08:30.672 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-20T18:08:30.672 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:08:30.672 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-20T18:08:30.673 INFO:tasks.workunit.client.0.trial195.stdout:adding osd0 key to auth repository 2026-02-20T18:08:30.673 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:08:30.776 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-20T18:08:30.776 INFO:tasks.workunit.client.0.trial195.stdout:start osd.0 2026-02-20T18:08:30.777 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:08:30.777 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:08:30.777 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:08:30.777 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:08:30.794 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:30.792+0000 7f538f5b8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:30.795 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:30.793+0000 7f538f5b8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:30.796 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:30.794+0000 7f538f5b8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:30.876 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-20T18:08:30.876 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:08:30.876 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-20T18:08:30.876 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:08:30.876 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:08:30.876 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:08:30.876 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:08:30.877 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:08:30.877 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:08:30.877 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T18:08:30.977 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:08:31.011 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:31.008+0000 7f538f5b8900 -1 Falling back to public interface 2026-02-20T18:08:31.264 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:31.262+0000 7f538f5b8900 -1 osd.0 0 log_to_monitors true 2026-02-20T18:08:31.978 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:08:31.978 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:08:31.978 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:08:31.979 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:08:31.979 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:08:31.979 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T18:08:32.203 INFO:tasks.workunit.client.0.trial195.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3775970211,v1:127.0.0.1:6803/3775970211] [v2:127.0.0.1:6804/3775970211,v1:127.0.0.1:6805/3775970211] exists,up 8187a2a4-6542-490d-b430-721555c01efb 2026-02-20T18:08:32.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:08:32.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:08:32.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:08:32.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:08:32.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-20T18:08:32.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:08:32.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:08:32.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-20T18:08:32.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:08:32.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-20T18:08:32.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:08:32.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:08:32.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:08:32.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:08:32.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-20T18:08:32.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-20T18:08:32.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:08:32.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:08:32.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:08:32.204 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:08:32.204 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:08:32.204 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:08:32.205 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:08:32.205 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:32.205 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:32.205 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:08:32.205 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:08:32.205 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:08:32.205 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:08:32.205 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:08:32.205 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:08:32.205 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:08:32.205 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:08:32.205 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:08:32.206 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:08:32.206 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:08:32.206 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:08:32.206 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:08:32.206 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-20T18:08:32.206 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:08:32.207 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=50c9db87-b5b8-4ada-b638-13e08cc547ac 2026-02-20T18:08:32.207 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 50c9db87-b5b8-4ada-b638-13e08cc547ac' 2026-02-20T18:08:32.207 INFO:tasks.workunit.client.0.trial195.stdout:add osd1 50c9db87-b5b8-4ada-b638-13e08cc547ac 2026-02-20T18:08:32.207 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:08:32.219 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAgo5hpiBcFDRAAJX59gkrQ3PP6PloTQS4PEg== 2026-02-20T18:08:32.219 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAgo5hpiBcFDRAAJX59gkrQ3PP6PloTQS4PEg=="}' 2026-02-20T18:08:32.219 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 50c9db87-b5b8-4ada-b638-13e08cc547ac -i td/test-erasure-eio/1/new.json 2026-02-20T18:08:32.449 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:08:32.457 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-20T18:08:32.458 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAgo5hpiBcFDRAAJX59gkrQ3PP6PloTQS4PEg== --osd-uuid 50c9db87-b5b8-4ada-b638-13e08cc547ac 2026-02-20T18:08:32.476 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:32.474+0000 7fa5c22b8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:32.477 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:32.476+0000 7fa5c22b8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:32.478 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:32.477+0000 7fa5c22b8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:32.479 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:32.477+0000 7fa5c22b8900 -1 bdev(0x5576c36cec00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-20T18:08:32.479 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:32.477+0000 7fa5c22b8900 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-20T18:08:32.812 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-20T18:08:32.812 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:08:32.813 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-20T18:08:32.813 INFO:tasks.workunit.client.0.trial195.stdout:adding osd1 key to auth repository 2026-02-20T18:08:32.814 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:08:33.131 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-20T18:08:33.131 INFO:tasks.workunit.client.0.trial195.stdout:start osd.1 2026-02-20T18:08:33.132 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:08:33.132 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:08:33.132 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:08:33.132 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:08:33.148 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:33.146+0000 7f65aa2b8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:33.149 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:33.147+0000 7f65aa2b8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:33.150 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:33.148+0000 7f65aa2b8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:33.289 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:33.287+0000 7f65aa2b8900 -1 Falling back to public interface 2026-02-20T18:08:33.361 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-20T18:08:33.361 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:08:33.361 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-20T18:08:33.362 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:08:33.362 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:08:33.362 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:08:33.362 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:08:33.362 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:08:33.363 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:08:33.363 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:08:33.586 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:08:33.621 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:33.619+0000 7f65aa2b8900 -1 osd.1 0 log_to_monitors true 2026-02-20T18:08:34.588 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:08:34.588 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:08:34.588 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:08:34.589 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:08:34.589 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:08:34.589 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:08:34.812 INFO:tasks.workunit.client.0.trial195.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2047762399,v1:127.0.0.1:6811/2047762399] [v2:127.0.0.1:6812/2047762399,v1:127.0.0.1:6813/2047762399] exists,up 50c9db87-b5b8-4ada-b638-13e08cc547ac 2026-02-20T18:08:34.815 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:08:34.816 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:08:34.816 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:08:34.816 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:08:34.816 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-20T18:08:34.816 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:08:34.816 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:08:34.816 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-20T18:08:34.816 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:08:34.816 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-20T18:08:34.816 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:08:34.817 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:08:34.817 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:08:34.817 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:08:34.817 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-20T18:08:34.817 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-20T18:08:34.817 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:08:34.817 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:08:34.817 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:08:34.817 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:08:34.817 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:08:34.817 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:08:34.817 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:08:34.818 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:34.818 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:34.818 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:08:34.818 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:08:34.818 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:08:34.818 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:08:34.818 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:08:34.818 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:08:34.818 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:08:34.818 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:08:34.818 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:08:34.819 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:08:34.819 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:08:34.819 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:08:34.819 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:08:34.819 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-20T18:08:34.819 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:08:34.820 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=85c2ec29-754a-419a-9075-46bb2c49919e 2026-02-20T18:08:34.820 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 85c2ec29-754a-419a-9075-46bb2c49919e' 2026-02-20T18:08:34.820 INFO:tasks.workunit.client.0.trial195.stdout:add osd2 85c2ec29-754a-419a-9075-46bb2c49919e 2026-02-20T18:08:34.820 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:08:34.828 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAio5hpxo1JMRAAKVE3k90OEeBK9Kozstu6sw== 2026-02-20T18:08:34.828 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAio5hpxo1JMRAAKVE3k90OEeBK9Kozstu6sw=="}' 2026-02-20T18:08:34.828 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 85c2ec29-754a-419a-9075-46bb2c49919e -i td/test-erasure-eio/2/new.json 2026-02-20T18:08:35.062 INFO:tasks.workunit.client.0.trial195.stdout:2 2026-02-20T18:08:35.070 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-20T18:08:35.071 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAio5hpxo1JMRAAKVE3k90OEeBK9Kozstu6sw== --osd-uuid 85c2ec29-754a-419a-9075-46bb2c49919e 2026-02-20T18:08:35.090 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:35.088+0000 7f54cdb20900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:35.091 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:35.090+0000 7f54cdb20900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:35.092 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:35.091+0000 7f54cdb20900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:35.093 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:35.091+0000 7f54cdb20900 -1 bdev(0x559fc0574c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-20T18:08:35.093 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:35.091+0000 7f54cdb20900 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-20T18:08:35.410 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-20T18:08:35.410 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:08:35.411 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-20T18:08:35.411 INFO:tasks.workunit.client.0.trial195.stdout:adding osd2 key to auth repository 2026-02-20T18:08:35.411 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:08:35.728 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-20T18:08:35.728 INFO:tasks.workunit.client.0.trial195.stdout:start osd.2 2026-02-20T18:08:35.729 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:08:35.729 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:08:35.729 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:08:35.729 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:08:35.749 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:35.747+0000 7fa722b18900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:35.750 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:35.749+0000 7fa722b18900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:35.751 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:35.749+0000 7fa722b18900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:35.866 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:35.864+0000 7fa722b18900 -1 Falling back to public interface 2026-02-20T18:08:35.953 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-20T18:08:35.953 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:08:35.953 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-20T18:08:35.953 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:08:35.953 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:08:35.953 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:08:35.953 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:08:35.953 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:08:35.954 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:08:35.954 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:08:36.182 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:08:36.183 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:36.182+0000 7fa722b18900 -1 osd.2 0 log_to_monitors true 2026-02-20T18:08:37.183 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:08:37.183 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:08:37.184 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:08:37.184 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:08:37.184 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:08:37.184 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:08:37.410 INFO:tasks.workunit.client.0.trial195.stdout:osd.2 up in weight 1 up_from 17 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2445115913,v1:127.0.0.1:6819/2445115913] [v2:127.0.0.1:6820/2445115913,v1:127.0.0.1:6821/2445115913] exists,up 85c2ec29-754a-419a-9075-46bb2c49919e 2026-02-20T18:08:37.417 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:08:37.417 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:08:37.418 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:08:37.418 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:08:37.418 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-20T18:08:37.418 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:08:37.418 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:08:37.418 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-20T18:08:37.418 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:08:37.419 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-20T18:08:37.419 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:08:37.419 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:08:37.419 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:08:37.419 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:08:37.419 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-20T18:08:37.420 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-20T18:08:37.420 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:08:37.420 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:08:37.420 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:08:37.420 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:08:37.420 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:08:37.420 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:08:37.421 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:08:37.421 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:37.421 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:37.421 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:08:37.421 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:08:37.421 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:08:37.421 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:08:37.422 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:08:37.422 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:08:37.422 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:08:37.422 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:08:37.422 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:08:37.422 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:08:37.422 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:08:37.423 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:08:37.423 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:08:37.423 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-20T18:08:37.423 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:08:37.423 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=31bacd4a-8cd5-4a51-b8ba-5ccfb0a225db 2026-02-20T18:08:37.423 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 31bacd4a-8cd5-4a51-b8ba-5ccfb0a225db' 2026-02-20T18:08:37.424 INFO:tasks.workunit.client.0.trial195.stdout:add osd3 31bacd4a-8cd5-4a51-b8ba-5ccfb0a225db 2026-02-20T18:08:37.424 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:08:37.426 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAlo5hpgEtOGRAATixlScIbuAXSy0a6bCyUoA== 2026-02-20T18:08:37.426 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAlo5hpgEtOGRAATixlScIbuAXSy0a6bCyUoA=="}' 2026-02-20T18:08:37.426 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 31bacd4a-8cd5-4a51-b8ba-5ccfb0a225db -i td/test-erasure-eio/3/new.json 2026-02-20T18:08:37.656 INFO:tasks.workunit.client.0.trial195.stdout:3 2026-02-20T18:08:37.664 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-20T18:08:37.665 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAlo5hpgEtOGRAATixlScIbuAXSy0a6bCyUoA== --osd-uuid 31bacd4a-8cd5-4a51-b8ba-5ccfb0a225db 2026-02-20T18:08:37.683 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:37.681+0000 7f9c9ee53900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:37.684 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:37.682+0000 7f9c9ee53900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:37.685 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:37.683+0000 7f9c9ee53900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:37.686 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:37.684+0000 7f9c9ee53900 -1 bdev(0x5640a42bac00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-20T18:08:37.686 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:37.684+0000 7f9c9ee53900 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-20T18:08:38.019 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-20T18:08:38.019 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:08:38.019 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-20T18:08:38.020 INFO:tasks.workunit.client.0.trial195.stdout:adding osd3 key to auth repository 2026-02-20T18:08:38.020 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:08:38.351 INFO:tasks.workunit.client.0.trial195.stdout:start osd.3 2026-02-20T18:08:38.351 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-20T18:08:38.351 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:08:38.351 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:08:38.351 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:08:38.351 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:08:38.368 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:38.366+0000 7f7374e0d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:38.370 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:38.368+0000 7f7374e0d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:38.371 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:38.369+0000 7f7374e0d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:38.501 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:38.498+0000 7f7374e0d900 -1 Falling back to public interface 2026-02-20T18:08:38.575 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-20T18:08:38.575 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:08:38.575 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-20T18:08:38.575 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:08:38.576 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:08:38.576 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:08:38.576 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:08:38.576 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:08:38.576 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:08:38.576 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T18:08:38.801 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:08:38.855 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:38.853+0000 7f7374e0d900 -1 osd.3 0 log_to_monitors true 2026-02-20T18:08:39.803 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:08:39.803 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:08:39.803 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:08:39.804 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:08:39.804 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:08:39.804 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T18:08:40.026 INFO:tasks.workunit.client.0.trial195.stdout:osd.3 up in weight 1 up_from 25 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3875962379,v1:127.0.0.1:6827/3875962379] [v2:127.0.0.1:6828/3875962379,v1:127.0.0.1:6829/3875962379] exists,up 31bacd4a-8cd5-4a51-b8ba-5ccfb0a225db 2026-02-20T18:08:40.028 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:08:40.028 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:08:40.028 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:08:40.028 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-20T18:08:40.028 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-20T18:08:40.028 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-20T18:08:40.028 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:08:40.028 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:40.028 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:40.029 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.0.asok 2026-02-20T18:08:40.029 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-20T18:08:40.029 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-osd.0.asok log flush 2026-02-20T18:08:40.073 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T18:08:40.077 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-20T18:08:40.079 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T18:08:31.011+0000 7f538f5b8900 0 load: jerasure load: lrc 2026-02-20T18:08:40.080 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:305: TEST_rados_get_subread_missing: local poolname=pool-jerasure 2026-02-20T18:08:40.080 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:306: TEST_rados_get_subread_missing: create_erasure_coded_pool pool-jerasure 2 1 2026-02-20T18:08:40.080 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-20T18:08:40.080 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-20T18:08:40.080 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-20T18:08:40.080 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-20T18:08:40.080 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-20T18:08:40.080 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-20T18:08:40.080 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-20T18:08:40.363 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-20T18:08:40.363 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-20T18:08:40.736 INFO:tasks.workunit.client.0.trial195.stderr:pool 'pool-jerasure' already exists 2026-02-20T18:08:40.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-20T18:08:41.745 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-20T18:08:41.745 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:08:41.745 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:08:41.745 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:08:41.745 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:08:41.746 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:08:41.746 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:08:41.746 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:08:41.747 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:08:41.747 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:08:41.835 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:08:41.835 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:08:41.835 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:08:41.835 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:08:41.835 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:08:41.835 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:08:42.061 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:08:42.062 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:08:42.062 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:08:42.062 INFO:tasks.workunit.client.0.trial195.stderr:3' 2026-02-20T18:08:42.062 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:08:42.062 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:08:42.062 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:08:42.127 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803781 2026-02-20T18:08:42.128 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803781 2026-02-20T18:08:42.128 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781' 2026-02-20T18:08:42.128 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:08:42.128 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:08:42.190 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607556 2026-02-20T18:08:42.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607556 2026-02-20T18:08:42.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781 1-51539607556' 2026-02-20T18:08:42.191 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:08:42.191 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:08:42.255 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444036 2026-02-20T18:08:42.255 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444036 2026-02-20T18:08:42.255 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781 1-51539607556 2-73014444036' 2026-02-20T18:08:42.255 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:08:42.255 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:08:42.318 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182403 2026-02-20T18:08:42.318 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182403 2026-02-20T18:08:42.318 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781 1-51539607556 2-73014444036 3-107374182403' 2026-02-20T18:08:42.318 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:08:42.319 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803781 2026-02-20T18:08:42.319 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:08:42.320 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:08:42.320 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803781 2026-02-20T18:08:42.321 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:08:42.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803781 2026-02-20T18:08:42.322 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-20T18:08:42.322 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.0 seq 25769803781 2026-02-20T18:08:42.322 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:08:42.546 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803779 -lt 25769803781 2026-02-20T18:08:42.546 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:08:43.547 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:08:43.548 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:08:43.775 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803782 -lt 25769803781 2026-02-20T18:08:43.775 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:08:43.775 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607556 2026-02-20T18:08:43.775 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:08:43.776 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:08:43.777 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-51539607556 2026-02-20T18:08:43.777 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:08:43.778 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=51539607556 2026-02-20T18:08:43.778 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 51539607556' 2026-02-20T18:08:43.779 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.1 seq 51539607556 2026-02-20T18:08:43.779 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:08:44.003 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 51539607556 -lt 51539607556 2026-02-20T18:08:44.003 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:08:44.003 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444036 2026-02-20T18:08:44.004 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:08:44.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:08:44.005 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-73014444036 2026-02-20T18:08:44.005 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:08:44.006 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=73014444036 2026-02-20T18:08:44.007 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 73014444036' 2026-02-20T18:08:44.007 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.2 seq 73014444036 2026-02-20T18:08:44.008 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:08:44.231 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 73014444037 -lt 73014444036 2026-02-20T18:08:44.231 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:08:44.231 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182403 2026-02-20T18:08:44.231 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:08:44.232 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:08:44.233 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-107374182403 2026-02-20T18:08:44.233 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:08:44.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=107374182403 2026-02-20T18:08:44.234 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 107374182403' 2026-02-20T18:08:44.234 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.3 seq 107374182403 2026-02-20T18:08:44.234 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:08:44.459 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 107374182403 -lt 107374182403 2026-02-20T18:08:44.460 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:08:44.460 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:08:44.460 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:08:44.773 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:08:44.773 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:08:44.773 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:08:44.773 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:08:44.773 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:08:44.773 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:08:44.774 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:08:44.774 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:08:44.997 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:08:44.997 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:08:44.998 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:08:44.998 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:08:45.306 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:08:45.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:08:45.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:08:45.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:308: TEST_rados_get_subread_missing: local shard_id=1 2026-02-20T18:08:45.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:309: TEST_rados_get_subread_missing: rados_put_get_data remove td/test-erasure-eio 1 2026-02-20T18:08:45.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=remove 2026-02-20T18:08:45.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-20T18:08:45.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-20T18:08:45.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-20T18:08:45.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=1 2026-02-20T18:08:45.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-20T18:08:45.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg= 2026-02-20T18:08:45.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-20T18:08:45.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-remove-85808-1 2026-02-20T18:08:45.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-remove-85808-1 2026-02-20T18:08:45.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-20T18:08:45.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-20T18:08:45.307 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-remove-85808-1 2026-02-20T18:08:45.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:08:45.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-20T18:08:45.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:08:45.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-20T18:08:45.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:08:45.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-20T18:08:45.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:08:45.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-20T18:08:45.308 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-remove-85808-1 td/test-erasure-eio/ORIGINAL 2026-02-20T18:08:45.327 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_remove ec data pool-jerasure obj-remove-85808-1 td/test-erasure-eio 1 2026-02-20T18:08:45.328 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:127: inject_remove: local pooltype=ec 2026-02-20T18:08:45.328 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:128: inject_remove: shift 2026-02-20T18:08:45.328 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:129: inject_remove: local which=data 2026-02-20T18:08:45.328 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:130: inject_remove: shift 2026-02-20T18:08:45.328 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:131: inject_remove: local poolname=pool-jerasure 2026-02-20T18:08:45.328 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:132: inject_remove: shift 2026-02-20T18:08:45.328 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:133: inject_remove: local objname=obj-remove-85808-1 2026-02-20T18:08:45.328 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:134: inject_remove: shift 2026-02-20T18:08:45.328 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:135: inject_remove: local dir=td/test-erasure-eio 2026-02-20T18:08:45.329 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:136: inject_remove: shift 2026-02-20T18:08:45.329 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:137: inject_remove: local shard_id=1 2026-02-20T18:08:45.329 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:138: inject_remove: shift 2026-02-20T18:08:45.329 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: get_osds pool-jerasure obj-remove-85808-1 2026-02-20T18:08:45.329 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:08:45.329 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-remove-85808-1 2026-02-20T18:08:45.329 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-remove-85808-1 2026-02-20T18:08:45.329 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:08:45.552 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T18:08:45.552 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:08:45.553 INFO:tasks.workunit.client.0.trial195.stderr:2' 2026-02-20T18:08:45.553 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-20T18:08:45.553 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: initial_osds=('3' '1' '2') 2026-02-20T18:08:45.553 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: local -a initial_osds 2026-02-20T18:08:45.553 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:141: inject_remove: local osd_id=1 2026-02-20T18:08:45.553 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:142: inject_remove: objectstore_tool td/test-erasure-eio 1 obj-remove-85808-1 remove 2026-02-20T18:08:45.553 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-20T18:08:45.553 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-20T18:08:45.553 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-20T18:08:45.553 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-20T18:08:45.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-remove-85808-1 remove 2026-02-20T18:08:45.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-20T18:08:45.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-20T18:08:45.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-20T18:08:45.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-20T18:08:45.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-20T18:08:45.554 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:08:45.554 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:08:45.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:08:45.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:08:45.554 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:08:45.858 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:08:45.859 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-remove-85808-1 remove 2026-02-20T18:08:45.859 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-20T18:08:45.859 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-20T18:08:45.859 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-20T18:08:45.859 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-20T18:08:45.859 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-20T18:08:45.859 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-remove-85808-1 remove 2026-02-20T18:08:45.958 INFO:tasks.workunit.client.0.trial195.stdout:remove 1#2:816538fa:::obj-remove-85808-1:head# 2026-02-20T18:08:45.995 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-20T18:08:45.995 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-20T18:08:45.995 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-20T18:08:45.995 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-20T18:08:45.995 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-20T18:08:45.995 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-20T18:08:45.995 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:08:45.995 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:08:45.995 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:08:45.995 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:08:45.995 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-20T18:08:45.995 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-20T18:08:45.995 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-20T18:08:45.995 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-20T18:08:45.995 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:08:45.995 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-20T18:08:45.995 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:08:45.995 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:08:45.995 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:08:45.996 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:45.996 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:45.996 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:08:45.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:08:45.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:08:45.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:08:45.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:08:45.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:08:45.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:08:45.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:08:45.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:08:45.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-20T18:08:45.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-20T18:08:45.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-20T18:08:45.996 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-20T18:08:45.996 INFO:tasks.workunit.client.0.trial195.stderr:start osd.1 2026-02-20T18:08:45.997 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:08:45.997 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-20T18:08:45.998 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-20T18:08:45.998 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-20T18:08:45.998 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-20T18:08:45.998 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-20T18:08:46.014 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:46.012+0000 7f8b19ef5900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:46.015 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:46.013+0000 7f8b19ef5900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:46.016 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:46.014+0000 7f8b19ef5900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:46.148 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:46.146+0000 7f8b19ef5900 -1 Falling back to public interface 2026-02-20T18:08:46.225 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-20T18:08:46.225 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:08:46.225 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-20T18:08:46.226 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:08:46.226 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:08:46.226 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:08:46.226 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:08:46.226 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T18:08:46.226 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:08:46.226 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:08:46.446 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:08:46.533 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:46.531+0000 7f8b19ef5900 -1 osd.1 33 log_to_monitors true 2026-02-20T18:08:47.448 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:08:47.448 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:08:47.448 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:08:47.448 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:08:47.448 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:08:47.448 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:08:47.677 INFO:tasks.workunit.client.0.trial195.stderr:osd.1 up in weight 1 up_from 37 up_thru 37 down_at 34 last_clean_interval [12,33) [v2:127.0.0.1:6810/343347620,v1:127.0.0.1:6811/343347620] [v2:127.0.0.1:6812/343347620,v1:127.0.0.1:6813/343347620] exists,up 50c9db87-b5b8-4ada-b638-13e08cc547ac 2026-02-20T18:08:47.677 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:08:47.677 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:08:47.677 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:08:47.677 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-20T18:08:47.677 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:08:47.677 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:08:47.677 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:08:47.678 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:08:47.678 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:08:47.678 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:08:47.678 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:08:47.678 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:08:47.678 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:08:47.762 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:08:47.762 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:08:47.763 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:08:47.763 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:08:47.763 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:08:47.763 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:08:47.988 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:08:47.988 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:08:47.988 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:08:47.988 INFO:tasks.workunit.client.0.trial195.stderr:3' 2026-02-20T18:08:47.988 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:08:47.988 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:08:47.989 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:08:48.052 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803785 2026-02-20T18:08:48.052 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803785 2026-02-20T18:08:48.052 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803785' 2026-02-20T18:08:48.052 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:08:48.052 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:08:48.118 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=158913789955 2026-02-20T18:08:48.118 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 158913789955 2026-02-20T18:08:48.118 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803785 1-158913789955' 2026-02-20T18:08:48.118 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:08:48.118 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:08:48.184 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444040 2026-02-20T18:08:48.184 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444040 2026-02-20T18:08:48.184 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803785 1-158913789955 2-73014444040' 2026-02-20T18:08:48.185 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:08:48.185 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:08:48.249 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182406 2026-02-20T18:08:48.249 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182406 2026-02-20T18:08:48.249 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803785 1-158913789955 2-73014444040 3-107374182406' 2026-02-20T18:08:48.249 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:08:48.249 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803785 2026-02-20T18:08:48.249 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:08:48.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:08:48.250 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803785 2026-02-20T18:08:48.251 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:08:48.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803785 2026-02-20T18:08:48.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803785' 2026-02-20T18:08:48.252 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.0 seq 25769803785 2026-02-20T18:08:48.252 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:08:48.482 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803782 -lt 25769803785 2026-02-20T18:08:48.482 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:08:49.483 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:08:49.484 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:08:49.705 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803785 -lt 25769803785 2026-02-20T18:08:49.706 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:08:49.706 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-158913789955 2026-02-20T18:08:49.706 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:08:49.707 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:08:49.708 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-158913789955 2026-02-20T18:08:49.708 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:08:49.709 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=158913789955 2026-02-20T18:08:49.709 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 158913789955' 2026-02-20T18:08:49.709 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.1 seq 158913789955 2026-02-20T18:08:49.710 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:08:49.934 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 158913789955 -lt 158913789955 2026-02-20T18:08:49.934 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:08:49.935 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444040 2026-02-20T18:08:49.935 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:08:49.936 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:08:49.936 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-73014444040 2026-02-20T18:08:49.936 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:08:49.937 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=73014444040 2026-02-20T18:08:49.937 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 73014444040' 2026-02-20T18:08:49.937 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.2 seq 73014444040 2026-02-20T18:08:49.938 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:08:50.161 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 73014444040 -lt 73014444040 2026-02-20T18:08:50.161 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:08:50.161 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182406 2026-02-20T18:08:50.161 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:08:50.162 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:08:50.163 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-107374182406 2026-02-20T18:08:50.163 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:08:50.164 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=107374182406 2026-02-20T18:08:50.164 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 107374182406' 2026-02-20T18:08:50.164 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.3 seq 107374182406 2026-02-20T18:08:50.165 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:08:50.389 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 107374182406 -lt 107374182406 2026-02-20T18:08:50.390 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:08:50.390 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:08:50.390 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:08:50.702 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:08:50.702 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:08:50.702 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:08:50.702 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:08:50.702 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:08:50.702 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:08:50.702 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:08:50.703 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:08:50.923 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:08:50.923 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:08:50.924 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:08:50.924 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:08:51.236 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:08:51.236 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:08:51.236 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:08:51.236 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-remove-85808-1 2026-02-20T18:08:51.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-20T18:08:51.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-20T18:08:51.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-remove-85808-1 2026-02-20T18:08:51.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-20T18:08:51.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-20T18:08:51.237 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-remove-85808-1 td/test-erasure-eio/COPY 2026-02-20T18:08:51.257 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-20T18:08:51.258 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-20T18:08:51.259 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' '' = recovery ']' 2026-02-20T18:08:51.259 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: expr 1 + 1 2026-02-20T18:08:51.261 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: local another_shard_id=2 2026-02-20T18:08:51.261 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:182: rados_put_get_data: inject_remove ec data pool-jerasure obj-remove-85808-1 td/test-erasure-eio 2 2026-02-20T18:08:51.261 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:127: inject_remove: local pooltype=ec 2026-02-20T18:08:51.261 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:128: inject_remove: shift 2026-02-20T18:08:51.261 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:129: inject_remove: local which=data 2026-02-20T18:08:51.261 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:130: inject_remove: shift 2026-02-20T18:08:51.261 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:131: inject_remove: local poolname=pool-jerasure 2026-02-20T18:08:51.261 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:132: inject_remove: shift 2026-02-20T18:08:51.261 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:133: inject_remove: local objname=obj-remove-85808-1 2026-02-20T18:08:51.262 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:134: inject_remove: shift 2026-02-20T18:08:51.262 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:135: inject_remove: local dir=td/test-erasure-eio 2026-02-20T18:08:51.262 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:136: inject_remove: shift 2026-02-20T18:08:51.262 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:137: inject_remove: local shard_id=2 2026-02-20T18:08:51.262 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:138: inject_remove: shift 2026-02-20T18:08:51.262 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: get_osds pool-jerasure obj-remove-85808-1 2026-02-20T18:08:51.262 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:08:51.262 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-remove-85808-1 2026-02-20T18:08:51.262 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-remove-85808-1 2026-02-20T18:08:51.262 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:08:51.491 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T18:08:51.491 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:08:51.491 INFO:tasks.workunit.client.0.trial195.stderr:2' 2026-02-20T18:08:51.491 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-20T18:08:51.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: initial_osds=('3' '1' '2') 2026-02-20T18:08:51.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: local -a initial_osds 2026-02-20T18:08:51.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:141: inject_remove: local osd_id=2 2026-02-20T18:08:51.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:142: inject_remove: objectstore_tool td/test-erasure-eio 2 obj-remove-85808-1 remove 2026-02-20T18:08:51.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-20T18:08:51.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-20T18:08:51.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-20T18:08:51.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-20T18:08:51.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-remove-85808-1 remove 2026-02-20T18:08:51.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-20T18:08:51.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-20T18:08:51.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-20T18:08:51.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-20T18:08:51.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-20T18:08:51.492 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:08:51.492 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:08:51.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:08:51.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:08:51.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:08:51.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:08:51.799 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-remove-85808-1 remove 2026-02-20T18:08:51.799 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-20T18:08:51.799 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-20T18:08:51.799 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-20T18:08:51.799 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-20T18:08:51.799 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-20T18:08:51.799 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-remove-85808-1 remove 2026-02-20T18:08:51.900 INFO:tasks.workunit.client.0.trial195.stdout:remove 2#2:816538fa:::obj-remove-85808-1:head# 2026-02-20T18:08:51.940 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-20T18:08:51.940 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-20T18:08:51.940 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-20T18:08:51.940 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-20T18:08:51.940 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-20T18:08:51.940 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-20T18:08:51.940 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:08:51.940 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:08:51.941 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:08:51.941 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:08:51.941 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-20T18:08:51.941 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-20T18:08:51.941 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-20T18:08:51.941 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-20T18:08:51.941 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:08:51.941 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-20T18:08:51.941 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:08:51.941 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:08:51.941 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:08:51.941 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:51.941 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:51.942 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:08:51.942 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:08:51.942 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:08:51.942 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:08:51.942 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:08:51.942 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:08:51.942 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:08:51.942 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:08:51.942 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:08:51.942 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-20T18:08:51.942 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-20T18:08:51.942 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-20T18:08:51.942 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-20T18:08:51.942 INFO:tasks.workunit.client.0.trial195.stderr:start osd.2 2026-02-20T18:08:51.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:08:51.943 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-20T18:08:51.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-20T18:08:51.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-20T18:08:51.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-20T18:08:51.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-20T18:08:51.959 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:51.958+0000 7efc6c60e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:51.960 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:51.959+0000 7efc6c60e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:51.961 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:51.960+0000 7efc6c60e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:52.092 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:52.091+0000 7efc6c60e900 -1 Falling back to public interface 2026-02-20T18:08:52.171 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-20T18:08:52.172 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:08:52.172 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-20T18:08:52.172 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:08:52.172 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:08:52.172 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:08:52.172 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:08:52.172 INFO:tasks.workunit.client.0.trial195.stderr:0 2026-02-20T18:08:52.172 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:08:52.172 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:08:52.392 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:08:52.438 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:52.437+0000 7efc6c60e900 -1 osd.2 40 log_to_monitors true 2026-02-20T18:08:53.394 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:08:53.394 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:08:53.394 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:08:53.394 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:08:53.394 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:08:53.394 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:08:53.624 INFO:tasks.workunit.client.0.trial195.stderr:osd.2 up in weight 1 up_from 43 up_thru 34 down_at 40 last_clean_interval [17,40) [v2:127.0.0.1:6818/3621539621,v1:127.0.0.1:6819/3621539621] [v2:127.0.0.1:6820/3621539621,v1:127.0.0.1:6821/3621539621] exists,up 85c2ec29-754a-419a-9075-46bb2c49919e 2026-02-20T18:08:53.625 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:08:53.625 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:08:53.625 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:08:53.625 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-20T18:08:53.625 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:08:53.625 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:08:53.625 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:08:53.625 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:08:53.625 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:08:53.626 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:08:53.626 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:08:53.626 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:08:53.626 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:08:53.713 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:08:53.714 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:08:53.714 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:08:53.714 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:08:53.714 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:08:53.714 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:08:53.940 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:08:53.940 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:08:53.940 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:08:53.940 INFO:tasks.workunit.client.0.trial195.stderr:3' 2026-02-20T18:08:53.941 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:08:53.941 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:08:53.941 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:08:54.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803788 2026-02-20T18:08:54.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803788 2026-02-20T18:08:54.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803788' 2026-02-20T18:08:54.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:08:54.004 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:08:54.071 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=158913789958 2026-02-20T18:08:54.071 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 158913789958 2026-02-20T18:08:54.071 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803788 1-158913789958' 2026-02-20T18:08:54.071 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:08:54.072 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:08:54.135 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593731 2026-02-20T18:08:54.135 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593731 2026-02-20T18:08:54.135 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803788 1-158913789958 2-184683593731' 2026-02-20T18:08:54.135 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:08:54.135 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:08:54.200 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182409 2026-02-20T18:08:54.200 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182409 2026-02-20T18:08:54.200 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803788 1-158913789958 2-184683593731 3-107374182409' 2026-02-20T18:08:54.200 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:08:54.201 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803788 2026-02-20T18:08:54.201 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:08:54.202 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:08:54.203 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803788 2026-02-20T18:08:54.203 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:08:54.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803788 2026-02-20T18:08:54.204 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803788' 2026-02-20T18:08:54.204 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.0 seq 25769803788 2026-02-20T18:08:54.204 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:08:54.427 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803786 -lt 25769803788 2026-02-20T18:08:54.427 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:08:55.427 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:08:55.428 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:08:55.654 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803788 -lt 25769803788 2026-02-20T18:08:55.654 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:08:55.654 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-158913789958 2026-02-20T18:08:55.654 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:08:55.655 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:08:55.656 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-158913789958 2026-02-20T18:08:55.656 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:08:55.657 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=158913789958 2026-02-20T18:08:55.657 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 158913789958' 2026-02-20T18:08:55.658 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.1 seq 158913789958 2026-02-20T18:08:55.658 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:08:55.883 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 158913789958 -lt 158913789958 2026-02-20T18:08:55.884 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:08:55.884 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-184683593731 2026-02-20T18:08:55.884 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:08:55.885 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:08:55.885 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-184683593731 2026-02-20T18:08:55.885 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:08:55.886 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=184683593731 2026-02-20T18:08:55.887 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 184683593731' 2026-02-20T18:08:55.887 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.2 seq 184683593731 2026-02-20T18:08:55.887 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:08:56.112 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 184683593731 -lt 184683593731 2026-02-20T18:08:56.112 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:08:56.113 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182409 2026-02-20T18:08:56.113 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:08:56.113 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:08:56.114 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-107374182409 2026-02-20T18:08:56.114 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:08:56.115 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=107374182409 2026-02-20T18:08:56.115 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 107374182409' 2026-02-20T18:08:56.115 INFO:tasks.workunit.client.0.trial195.stderr:waiting osd.3 seq 107374182409 2026-02-20T18:08:56.116 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:08:56.347 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 107374182409 -lt 107374182409 2026-02-20T18:08:56.348 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:08:56.348 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:08:56.348 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:08:56.662 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:08:56.662 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:08:56.662 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:08:56.663 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:08:56.663 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:08:56.663 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:08:56.663 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:08:56.663 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:08:56.885 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:08:56.885 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:08:56.885 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:08:56.885 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:08:57.195 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:08:57.195 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:08:57.195 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:08:57.195 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:183: rados_put_get_data: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-20T18:08:57.195 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:188: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-remove-85808-1 2026-02-20T18:08:57.195 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-20T18:08:57.195 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-20T18:08:57.195 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-remove-85808-1 2026-02-20T18:08:57.195 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-20T18:08:57.195 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-20T18:08:57.195 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-remove-85808-1 td/test-erasure-eio/COPY 2026-02-20T18:08:57.215 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-20T18:08:57.216 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-20T18:08:57.217 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:194: rados_put_get_data: rm td/test-erasure-eio/ORIGINAL 2026-02-20T18:08:57.217 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:310: TEST_rados_get_subread_missing: delete_erasure_coded_pool pool-jerasure 2026-02-20T18:08:57.218 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-20T18:08:57.218 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-20T18:08:57.491 INFO:tasks.workunit.client.0.trial195.stderr:pool 'pool-jerasure' does not exist 2026-02-20T18:08:57.498 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-20T18:08:57.775 INFO:tasks.workunit.client.0.trial195.stderr:erasure-code-profile myprofile does not exist 2026-02-20T18:08:57.783 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-20T18:08:57.783 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-20T18:08:57.783 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T18:08:57.783 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-20T18:08:57.784 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:08:57.784 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:08:57.784 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:08:57.784 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:08:57.784 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:08:57.893 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:08:57.894 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T18:08:57.895 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T18:08:57.896 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T18:08:57.897 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T18:08:57.897 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T18:08:57.897 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T18:08:57.898 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:08:57.898 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T18:08:57.899 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T18:08:57.899 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:08:57.900 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T18:08:57.900 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T18:08:57.900 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-20T18:08:57.911 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T18:08:57.911 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:57.911 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:57.911 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.85808 2026-02-20T18:08:57.913 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T18:08:57.913 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T18:08:57.913 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-20T18:08:57.913 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-20T18:08:57.913 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-20T18:08:57.913 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-20T18:08:57.913 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-20T18:08:57.914 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T18:08:57.914 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-20T18:08:57.914 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:08:57.914 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:08:57.914 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:08:57.914 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:08:57.914 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:08:57.914 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:08:57.915 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T18:08:57.915 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T18:08:57.916 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T18:08:57.917 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T18:08:57.917 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T18:08:57.917 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T18:08:57.918 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:08:57.918 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T18:08:57.919 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T18:08:57.919 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:08:57.919 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T18:08:57.920 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T18:08:57.920 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-20T18:08:57.921 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T18:08:57.921 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:57.921 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:57.921 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.85808 2026-02-20T18:08:57.922 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T18:08:57.922 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T18:08:57.922 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-20T18:08:57.924 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-20T18:08:57.924 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:57.924 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:57.924 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.85808 2026-02-20T18:08:57.925 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-20T18:08:57.925 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-20T18:08:57.926 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-20T18:08:57.926 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-20T18:08:57.926 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-20T18:08:57.926 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-20T18:08:57.926 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-20T18:08:57.926 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-20T18:08:57.926 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-20T18:08:57.926 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-20T18:08:57.926 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-20T18:08:57.949 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-20T18:08:57.949 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:08:57.949 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:08:57.949 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:08:57.949 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:57.949 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:57.949 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:08:57.950 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-20T18:08:57.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-20T18:08:57.974 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-20T18:08:57.974 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T18:08:57.974 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T18:08:57.975 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-20T18:08:57.975 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-20T18:08:57.975 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T18:08:57.975 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:08:57.975 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:08:57.975 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:08:57.975 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:57.975 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:57.976 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:08:57.976 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T18:08:57.976 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.85808/ceph-mon.a.asok config get fsid 2026-02-20T18:08:58.019 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-20T18:08:58.019 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T18:08:58.019 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T18:08:58.019 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-20T18:08:58.020 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-20T18:08:58.020 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T18:08:58.020 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:08:58.020 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:08:58.020 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:08:58.020 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:58.020 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:58.020 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:08:58.020 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T18:08:58.020 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.85808/ceph-mon.a.asok config get mon_host 2026-02-20T18:08:58.065 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-20T18:08:58.065 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-20T18:08:58.065 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-20T18:08:58.066 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-20T18:08:58.066 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-20T18:08:58.066 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-20T18:08:58.066 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-20T18:08:58.170 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-20T18:08:58.170 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:08:58.170 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:08:58.171 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:08:58.171 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:58.171 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:58.171 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:08:58.171 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T18:08:58.172 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T18:08:58.190 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-20T18:08:58.190 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-20T18:08:58.286 INFO:tasks.workunit.client.0.trial195.stderr:pool 'rbd' already exists 2026-02-20T18:08:58.292 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-20T18:08:59.294 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-20T18:08:59.294 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:08:59.294 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:08:59.295 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:08:59.295 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:59.295 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:59.295 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:08:59.295 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-20T18:08:59.295 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-mon.a.asok log flush 2026-02-20T18:08:59.337 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T18:08:59.342 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-20T18:08:59.343 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T18:08:57.966+0000 7fd8bd7dbd80 0 load: jerasure load: lrc 2026-02-20T18:08:59.343 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_with_subreadall_eio_shard_0 td/test-erasure-eio 2026-02-20T18:08:59.343 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:350: TEST_rados_get_with_subreadall_eio_shard_0: local dir=td/test-erasure-eio 2026-02-20T18:08:59.343 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:351: TEST_rados_get_with_subreadall_eio_shard_0: local shard_id=0 2026-02-20T18:08:59.343 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:353: TEST_rados_get_with_subreadall_eio_shard_0: setup_osds 4 2026-02-20T18:08:59.343 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-20T18:08:59.343 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-20T18:08:59.344 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-20T18:08:59.345 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-20T18:08:59.345 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:08:59.345 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-20T18:08:59.345 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:08:59.345 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:08:59.345 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-20T18:08:59.346 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:08:59.346 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-20T18:08:59.346 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:08:59.346 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:08:59.346 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:08:59.346 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:08:59.346 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-20T18:08:59.346 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-20T18:08:59.346 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:08:59.346 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:08:59.346 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:08:59.346 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:08:59.346 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:08:59.346 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:08:59.346 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:08:59.346 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:08:59.347 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:08:59.347 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:08:59.347 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:08:59.347 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:08:59.347 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:08:59.347 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:08:59.347 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:08:59.347 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:08:59.347 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:08:59.347 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:08:59.347 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:08:59.347 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:08:59.347 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:08:59.347 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:08:59.347 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-20T18:08:59.348 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:08:59.348 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5ef63a2f-c6e8-4077-b492-aeb9aa84dc36 2026-02-20T18:08:59.348 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 5ef63a2f-c6e8-4077-b492-aeb9aa84dc36' 2026-02-20T18:08:59.348 INFO:tasks.workunit.client.0.trial195.stdout:add osd0 5ef63a2f-c6e8-4077-b492-aeb9aa84dc36 2026-02-20T18:08:59.348 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:08:59.362 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA7o5hpwM5+FRAAxJvoRt59lhMgW5R0XVZxkQ== 2026-02-20T18:08:59.362 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA7o5hpwM5+FRAAxJvoRt59lhMgW5R0XVZxkQ=="}' 2026-02-20T18:08:59.362 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5ef63a2f-c6e8-4077-b492-aeb9aa84dc36 -i td/test-erasure-eio/0/new.json 2026-02-20T18:08:59.466 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:08:59.473 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-20T18:08:59.474 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA7o5hpwM5+FRAAxJvoRt59lhMgW5R0XVZxkQ== --osd-uuid 5ef63a2f-c6e8-4077-b492-aeb9aa84dc36 2026-02-20T18:08:59.491 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:59.490+0000 7f0922e53900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:59.493 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:59.492+0000 7f0922e53900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:59.493 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:59.493+0000 7f0922e53900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:08:59.494 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:59.493+0000 7f0922e53900 -1 bdev(0x559de73cec00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-20T18:08:59.494 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:08:59.493+0000 7f0922e53900 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-20T18:08:59.941 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-20T18:08:59.941 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:08:59.942 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-20T18:08:59.942 INFO:tasks.workunit.client.0.trial195.stdout:adding osd0 key to auth repository 2026-02-20T18:08:59.942 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:09:00.049 INFO:tasks.workunit.client.0.trial195.stdout:start osd.0 2026-02-20T18:09:00.049 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-20T18:09:00.049 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:09:00.049 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:09:00.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:09:00.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:09:00.067 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:00.066+0000 7fa7abad9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:00.068 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:00.067+0000 7fa7abad9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:00.069 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:00.068+0000 7fa7abad9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:00.149 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-20T18:09:00.149 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:09:00.149 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-20T18:09:00.149 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:09:00.149 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:09:00.149 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:09:00.149 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:09:00.149 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:09:00.149 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:09:00.149 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T18:09:00.248 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:09:00.271 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:00.271+0000 7fa7abad9900 -1 Falling back to public interface 2026-02-20T18:09:00.527 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:00.526+0000 7fa7abad9900 -1 osd.0 0 log_to_monitors true 2026-02-20T18:09:01.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:09:01.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:09:01.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:09:01.251 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:09:01.251 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:09:01.251 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T18:09:01.476 INFO:tasks.workunit.client.0.trial195.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1793460193,v1:127.0.0.1:6803/1793460193] [v2:127.0.0.1:6804/1793460193,v1:127.0.0.1:6805/1793460193] exists,up 5ef63a2f-c6e8-4077-b492-aeb9aa84dc36 2026-02-20T18:09:01.480 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:09:01.480 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:09:01.480 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:09:01.480 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:09:01.480 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-20T18:09:01.480 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:09:01.480 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:09:01.480 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-20T18:09:01.480 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:09:01.481 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-20T18:09:01.481 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:09:01.481 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:09:01.481 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:09:01.481 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:09:01.481 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-20T18:09:01.481 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-20T18:09:01.481 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:09:01.481 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:09:01.481 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:09:01.481 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:09:01.481 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:09:01.482 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:09:01.482 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:09:01.482 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:09:01.482 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:09:01.482 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:09:01.482 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:09:01.482 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:09:01.482 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:09:01.482 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:09:01.482 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:09:01.482 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:09:01.482 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:09:01.483 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:09:01.483 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:09:01.483 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:09:01.483 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:09:01.483 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:09:01.483 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-20T18:09:01.483 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:09:01.483 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=363e606a-821a-4884-a8e1-7a8e745d6acc 2026-02-20T18:09:01.483 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 363e606a-821a-4884-a8e1-7a8e745d6acc' 2026-02-20T18:09:01.483 INFO:tasks.workunit.client.0.trial195.stdout:add osd1 363e606a-821a-4884-a8e1-7a8e745d6acc 2026-02-20T18:09:01.484 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:09:01.491 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA9o5hpaG88HRAAZIaw5qzP6zjHVccuqn0Fcw== 2026-02-20T18:09:01.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA9o5hpaG88HRAAZIaw5qzP6zjHVccuqn0Fcw=="}' 2026-02-20T18:09:01.492 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 363e606a-821a-4884-a8e1-7a8e745d6acc -i td/test-erasure-eio/1/new.json 2026-02-20T18:09:01.719 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:09:01.727 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-20T18:09:01.727 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA9o5hpaG88HRAAZIaw5qzP6zjHVccuqn0Fcw== --osd-uuid 363e606a-821a-4884-a8e1-7a8e745d6acc 2026-02-20T18:09:01.745 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:01.744+0000 7f76b8823900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:01.747 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:01.746+0000 7f76b8823900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:01.748 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:01.747+0000 7f76b8823900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:01.748 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:01.747+0000 7f76b8823900 -1 bdev(0x5649a15f4c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-20T18:09:01.748 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:01.747+0000 7f76b8823900 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-20T18:09:02.069 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-20T18:09:02.069 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:09:02.069 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-20T18:09:02.070 INFO:tasks.workunit.client.0.trial195.stdout:adding osd1 key to auth repository 2026-02-20T18:09:02.070 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:09:02.397 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-20T18:09:02.398 INFO:tasks.workunit.client.0.trial195.stdout:start osd.1 2026-02-20T18:09:02.398 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:09:02.398 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:09:02.398 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:09:02.398 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:09:02.415 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:02.414+0000 7fdf4eba8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:02.416 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:02.415+0000 7fdf4eba8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:02.417 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:02.416+0000 7fdf4eba8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:02.547 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:02.546+0000 7fdf4eba8900 -1 Falling back to public interface 2026-02-20T18:09:02.624 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-20T18:09:02.624 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:09:02.624 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-20T18:09:02.624 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:09:02.624 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:09:02.624 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:09:02.624 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:09:02.624 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:09:02.624 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:09:02.624 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:09:02.851 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:09:02.853 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:02.853+0000 7fdf4eba8900 -1 osd.1 0 log_to_monitors true 2026-02-20T18:09:03.852 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:09:03.852 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:09:03.852 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:09:03.853 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:09:03.853 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:09:03.853 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:09:04.077 INFO:tasks.workunit.client.0.trial195.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2244027140,v1:127.0.0.1:6811/2244027140] [v2:127.0.0.1:6812/2244027140,v1:127.0.0.1:6813/2244027140] exists,up 363e606a-821a-4884-a8e1-7a8e745d6acc 2026-02-20T18:09:04.078 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:09:04.078 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:09:04.078 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:09:04.078 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:09:04.078 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-20T18:09:04.078 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:09:04.078 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:09:04.078 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-20T18:09:04.078 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:09:04.078 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-20T18:09:04.078 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:09:04.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:09:04.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:09:04.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:09:04.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-20T18:09:04.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-20T18:09:04.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:09:04.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:09:04.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:09:04.079 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:09:04.079 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:09:04.079 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:09:04.079 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:09:04.079 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:09:04.079 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:09:04.079 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:09:04.079 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:09:04.080 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:09:04.080 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:09:04.080 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:09:04.080 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:09:04.080 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:09:04.080 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:09:04.080 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:09:04.080 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:09:04.080 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:09:04.080 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:09:04.080 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:09:04.080 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-20T18:09:04.080 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:09:04.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=6d767e96-3c95-4464-92bf-dc0ea1ab44fb 2026-02-20T18:09:04.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 6d767e96-3c95-4464-92bf-dc0ea1ab44fb' 2026-02-20T18:09:04.081 INFO:tasks.workunit.client.0.trial195.stdout:add osd2 6d767e96-3c95-4464-92bf-dc0ea1ab44fb 2026-02-20T18:09:04.082 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:09:04.094 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBAo5hpAAmOBRAAOd2ww8G7GdsHHXfFZxy7Mg== 2026-02-20T18:09:04.094 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBAo5hpAAmOBRAAOd2ww8G7GdsHHXfFZxy7Mg=="}' 2026-02-20T18:09:04.094 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 6d767e96-3c95-4464-92bf-dc0ea1ab44fb -i td/test-erasure-eio/2/new.json 2026-02-20T18:09:04.317 INFO:tasks.workunit.client.0.trial195.stdout:2 2026-02-20T18:09:04.325 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-20T18:09:04.325 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBAo5hpAAmOBRAAOd2ww8G7GdsHHXfFZxy7Mg== --osd-uuid 6d767e96-3c95-4464-92bf-dc0ea1ab44fb 2026-02-20T18:09:04.344 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:04.343+0000 7f68d411a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:04.345 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:04.345+0000 7f68d411a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:04.346 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:04.346+0000 7f68d411a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:04.347 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:04.346+0000 7f68d411a900 -1 bdev(0x55db5fbbcc00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-20T18:09:04.347 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:04.346+0000 7f68d411a900 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-20T18:09:04.667 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-20T18:09:04.667 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:09:04.668 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-20T18:09:04.668 INFO:tasks.workunit.client.0.trial195.stdout:adding osd2 key to auth repository 2026-02-20T18:09:04.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:09:04.987 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-20T18:09:04.987 INFO:tasks.workunit.client.0.trial195.stdout:start osd.2 2026-02-20T18:09:04.988 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:09:04.988 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:09:04.988 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:09:04.989 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:09:05.003 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:05.002+0000 7f66d7c97900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:05.005 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:05.004+0000 7f66d7c97900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:05.006 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:05.005+0000 7f66d7c97900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:05.194 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:05.192+0000 7f66d7c97900 -1 Falling back to public interface 2026-02-20T18:09:05.211 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-20T18:09:05.211 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:09:05.211 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-20T18:09:05.211 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:09:05.211 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:09:05.211 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:09:05.211 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:09:05.212 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:09:05.212 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:09:05.212 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:09:05.439 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:09:05.558 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:05.557+0000 7f66d7c97900 -1 osd.2 0 log_to_monitors true 2026-02-20T18:09:06.440 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:09:06.441 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:09:06.441 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:09:06.441 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:09:06.441 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:09:06.441 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:09:06.668 INFO:tasks.workunit.client.0.trial195.stdout:osd.2 up in weight 1 up_from 19 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/627668165,v1:127.0.0.1:6819/627668165] [v2:127.0.0.1:6820/627668165,v1:127.0.0.1:6821/627668165] exists,up 6d767e96-3c95-4464-92bf-dc0ea1ab44fb 2026-02-20T18:09:06.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:09:06.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:09:06.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:09:06.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:09:06.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-20T18:09:06.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:09:06.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:09:06.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-20T18:09:06.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:09:06.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-20T18:09:06.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:09:06.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:09:06.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:09:06.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:09:06.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-20T18:09:06.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-20T18:09:06.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:09:06.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:09:06.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:09:06.670 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:09:06.670 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:09:06.670 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:09:06.670 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:09:06.671 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:09:06.671 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:09:06.671 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:09:06.671 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:09:06.671 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:09:06.671 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:09:06.671 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:09:06.671 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:09:06.671 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:09:06.671 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:09:06.671 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:09:06.671 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:09:06.671 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:09:06.671 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:09:06.671 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:09:06.671 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-20T18:09:06.671 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:09:06.672 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c2b9eeaf-1220-4e1d-bb35-b617d6622ff6 2026-02-20T18:09:06.672 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 c2b9eeaf-1220-4e1d-bb35-b617d6622ff6' 2026-02-20T18:09:06.672 INFO:tasks.workunit.client.0.trial195.stdout:add osd3 c2b9eeaf-1220-4e1d-bb35-b617d6622ff6 2026-02-20T18:09:06.673 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:09:06.686 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBCo5hptY7MKBAA7pWwrK0wfa3OvOg25KylkQ== 2026-02-20T18:09:06.686 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBCo5hptY7MKBAA7pWwrK0wfa3OvOg25KylkQ=="}' 2026-02-20T18:09:06.686 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c2b9eeaf-1220-4e1d-bb35-b617d6622ff6 -i td/test-erasure-eio/3/new.json 2026-02-20T18:09:06.914 INFO:tasks.workunit.client.0.trial195.stdout:3 2026-02-20T18:09:06.922 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-20T18:09:06.923 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBCo5hptY7MKBAA7pWwrK0wfa3OvOg25KylkQ== --osd-uuid c2b9eeaf-1220-4e1d-bb35-b617d6622ff6 2026-02-20T18:09:06.941 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:06.940+0000 7f3322b2d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:06.942 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:06.941+0000 7f3322b2d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:06.943 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:06.942+0000 7f3322b2d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:06.944 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:06.942+0000 7f3322b2d900 -1 bdev(0x5567a823ac00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-20T18:09:06.944 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:06.942+0000 7f3322b2d900 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-20T18:09:07.274 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-20T18:09:07.274 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:09:07.275 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-20T18:09:07.275 INFO:tasks.workunit.client.0.trial195.stdout:adding osd3 key to auth repository 2026-02-20T18:09:07.276 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:09:07.595 INFO:tasks.workunit.client.0.trial195.stdout:start osd.3 2026-02-20T18:09:07.596 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-20T18:09:07.596 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:09:07.596 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:09:07.596 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:09:07.596 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:09:07.611 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:07.611+0000 7f01050f7900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:07.613 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:07.612+0000 7f01050f7900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:07.614 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:07.613+0000 7f01050f7900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:07.778 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:07.777+0000 7f01050f7900 -1 Falling back to public interface 2026-02-20T18:09:07.825 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-20T18:09:07.825 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:09:07.825 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-20T18:09:07.825 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:09:07.825 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:09:07.825 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:09:07.825 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:09:07.825 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:09:07.825 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:09:07.825 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T18:09:08.056 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:09:08.095 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:08.094+0000 7f01050f7900 -1 osd.3 0 log_to_monitors true 2026-02-20T18:09:09.057 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:09:09.058 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:09:09.058 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:09:09.059 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:09:09.059 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:09:09.059 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T18:09:09.281 INFO:tasks.workunit.client.0.trial195.stdout:osd.3 up in weight 1 up_from 27 up_thru 28 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2893013867,v1:127.0.0.1:6827/2893013867] [v2:127.0.0.1:6828/2893013867,v1:127.0.0.1:6829/2893013867] exists,up c2b9eeaf-1220-4e1d-bb35-b617d6622ff6 2026-02-20T18:09:09.281 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:09:09.281 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:09:09.281 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:09:09.281 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-20T18:09:09.281 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-20T18:09:09.281 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-20T18:09:09.282 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:09:09.282 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:09:09.282 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:09:09.282 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.0.asok 2026-02-20T18:09:09.282 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-20T18:09:09.282 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-osd.0.asok log flush 2026-02-20T18:09:09.330 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T18:09:09.334 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-20T18:09:09.336 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T18:09:00.273+0000 7fa7abad9900 0 load: jerasure load: lrc 2026-02-20T18:09:09.336 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:355: TEST_rados_get_with_subreadall_eio_shard_0: local poolname=pool-jerasure 2026-02-20T18:09:09.336 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:356: TEST_rados_get_with_subreadall_eio_shard_0: create_erasure_coded_pool pool-jerasure 2 1 2026-02-20T18:09:09.336 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-20T18:09:09.336 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-20T18:09:09.336 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-20T18:09:09.336 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-20T18:09:09.337 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-20T18:09:09.337 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-20T18:09:09.337 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-20T18:09:09.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-20T18:09:09.620 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-20T18:09:09.991 INFO:tasks.workunit.client.0.trial195.stderr:pool 'pool-jerasure' already exists 2026-02-20T18:09:09.999 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-20T18:09:11.001 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-20T18:09:11.001 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:09:11.001 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:09:11.001 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:09:11.001 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:09:11.002 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:09:11.002 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:09:11.002 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:09:11.002 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:09:11.002 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:09:11.090 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:09:11.090 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:09:11.090 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:09:11.090 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:09:11.090 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:09:11.091 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:09:11.316 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:09:11.316 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:09:11.316 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:09:11.316 INFO:tasks.workunit.client.0.trial195.stderr:3' 2026-02-20T18:09:11.317 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:09:11.317 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:09:11.317 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:09:11.382 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803780 2026-02-20T18:09:11.382 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803780 2026-02-20T18:09:11.382 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803780' 2026-02-20T18:09:11.382 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:09:11.382 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:09:11.446 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574853 2026-02-20T18:09:11.446 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574853 2026-02-20T18:09:11.446 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803780 1-55834574853' 2026-02-20T18:09:11.446 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:09:11.446 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:09:11.513 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378627 2026-02-20T18:09:11.513 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378627 2026-02-20T18:09:11.513 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803780 1-55834574853 2-81604378627' 2026-02-20T18:09:11.513 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:09:11.513 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:09:11.576 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964116995 2026-02-20T18:09:11.576 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964116995 2026-02-20T18:09:11.576 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803780 1-55834574853 2-81604378627 3-115964116995' 2026-02-20T18:09:11.576 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:09:11.576 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803780 2026-02-20T18:09:11.576 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:09:11.577 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:09:11.578 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803780 2026-02-20T18:09:11.578 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:09:11.579 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803780 2026-02-20T18:09:11.579 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.0 seq 25769803780 2026-02-20T18:09:11.579 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803780' 2026-02-20T18:09:11.579 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:09:11.802 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803778 -lt 25769803780 2026-02-20T18:09:11.802 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:09:12.802 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:09:12.803 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:09:13.028 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803781 -lt 25769803780 2026-02-20T18:09:13.029 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:09:13.029 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574853 2026-02-20T18:09:13.029 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:09:13.030 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:09:13.031 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-55834574853 2026-02-20T18:09:13.031 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:09:13.032 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574853 2026-02-20T18:09:13.032 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 55834574853' 2026-02-20T18:09:13.032 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.1 seq 55834574853 2026-02-20T18:09:13.032 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:09:13.257 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574853 -lt 55834574853 2026-02-20T18:09:13.257 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:09:13.258 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-81604378627 2026-02-20T18:09:13.258 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:09:13.259 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:09:13.259 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-81604378627 2026-02-20T18:09:13.259 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:09:13.261 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=81604378627 2026-02-20T18:09:13.261 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 81604378627' 2026-02-20T18:09:13.261 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.2 seq 81604378627 2026-02-20T18:09:13.261 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:09:13.486 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 81604378628 -lt 81604378627 2026-02-20T18:09:13.486 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:09:13.486 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964116995 2026-02-20T18:09:13.486 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:09:13.486 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:09:13.487 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-115964116995 2026-02-20T18:09:13.487 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:09:13.488 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=115964116995 2026-02-20T18:09:13.488 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 115964116995' 2026-02-20T18:09:13.489 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.3 seq 115964116995 2026-02-20T18:09:13.489 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:09:13.711 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 115964116995 -lt 115964116995 2026-02-20T18:09:13.711 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:09:13.711 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:09:13.711 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:09:14.030 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:09:14.030 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:09:14.031 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:09:14.031 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:09:14.031 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:09:14.031 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:09:14.031 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:09:14.031 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:09:14.255 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:09:14.256 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:09:14.256 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:09:14.256 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:09:14.566 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:09:14.566 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:09:14.566 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:09:14.566 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:358: TEST_rados_get_with_subreadall_eio_shard_0: rados_put_get_data eio td/test-erasure-eio 0 recovery 2026-02-20T18:09:14.566 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=eio 2026-02-20T18:09:14.566 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-20T18:09:14.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-20T18:09:14.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-20T18:09:14.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=0 2026-02-20T18:09:14.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-20T18:09:14.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg=recovery 2026-02-20T18:09:14.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-20T18:09:14.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-eio-85808-0 2026-02-20T18:09:14.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-eio-85808-0 2026-02-20T18:09:14.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-20T18:09:14.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-20T18:09:14.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-eio-85808-0 2026-02-20T18:09:14.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:09:14.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-20T18:09:14.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:09:14.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-20T18:09:14.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:09:14.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-20T18:09:14.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:09:14.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-20T18:09:14.567 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-eio-85808-0 td/test-erasure-eio/ORIGINAL 2026-02-20T18:09:14.587 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-85808-0 td/test-erasure-eio 0 2026-02-20T18:09:14.587 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=ec 2026-02-20T18:09:14.587 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-20T18:09:14.587 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=data 2026-02-20T18:09:14.587 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-20T18:09:14.587 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=pool-jerasure 2026-02-20T18:09:14.587 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-20T18:09:14.587 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=obj-eio-85808-0 2026-02-20T18:09:14.587 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-20T18:09:14.587 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/test-erasure-eio 2026-02-20T18:09:14.587 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-20T18:09:14.587 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=0 2026-02-20T18:09:14.587 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-20T18:09:14.587 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds pool-jerasure obj-eio-85808-0 2026-02-20T18:09:14.587 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:09:14.587 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-85808-0 2026-02-20T18:09:14.587 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-85808-0 2026-02-20T18:09:14.587 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:09:14.817 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T18:09:14.817 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:09:14.817 INFO:tasks.workunit.client.0.trial195.stderr:2' 2026-02-20T18:09:14.817 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-20T18:09:14.818 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('3' '1' '2') 2026-02-20T18:09:14.818 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-20T18:09:14.818 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=3 2026-02-20T18:09:14.818 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' ec '!=' ec ']' 2026-02-20T18:09:14.818 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/test-erasure-eio/3/type 2026-02-20T18:09:14.818 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-20T18:09:14.819 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-20T18:09:14.819 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-20T18:09:14.819 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-20T18:09:14.819 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-20T18:09:14.819 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-20T18:09:14.819 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-20T18:09:14.819 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-20T18:09:14.819 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-20T18:09:14.820 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-20T18:09:14.820 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:09:14.820 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:09:14.820 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:09:14.820 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.3.asok 2026-02-20T18:09:14.820 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.85808/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-20T18:09:14.870 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-20T18:09:14.870 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-20T18:09:14.871 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-20T18:09:14.871 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.3 2026-02-20T18:09:14.872 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-20T18:09:14.872 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-20T18:09:14.872 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:09:14.872 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:09:14.872 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:09:14.872 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.3.asok 2026-02-20T18:09:14.872 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-20T18:09:14.872 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-osd.3.asok injectdataerr pool-jerasure obj-eio-85808-0 0 2026-02-20T18:09:14.921 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-85808-0 2026-02-20T18:09:14.921 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-20T18:09:14.921 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-20T18:09:14.921 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-85808-0 2026-02-20T18:09:14.921 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-20T18:09:14.921 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-20T18:09:14.921 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-85808-0 td/test-erasure-eio/COPY 2026-02-20T18:09:14.941 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-20T18:09:14.942 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-20T18:09:14.943 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' recovery = recovery ']' 2026-02-20T18:09:14.944 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: get_osds pool-jerasure obj-eio-85808-0 2026-02-20T18:09:14.944 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:09:14.944 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-85808-0 2026-02-20T18:09:14.944 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-85808-0 2026-02-20T18:09:14.944 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:09:15.171 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T18:09:15.171 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:09:15.171 INFO:tasks.workunit.client.0.trial195.stderr:2' 2026-02-20T18:09:15.171 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-20T18:09:15.171 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: initial_osds=('3' '1' '2') 2026-02-20T18:09:15.171 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: local -a initial_osds 2026-02-20T18:09:15.171 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:171: rados_put_get_data: local last_osd=2 2026-02-20T18:09:15.171 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:173: rados_put_get_data: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-20T18:09:15.172 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:09:15.172 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:09:15.172 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:09:15.172 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:09:15.172 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:09:15.479 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:09:15.479 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:174: rados_put_get_data: ceph osd out 2 2026-02-20T18:09:15.757 INFO:tasks.workunit.client.0.trial195.stderr:osd.2 is already out. 2026-02-20T18:09:15.765 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:175: rados_put_get_data: get_osds pool-jerasure obj-eio-85808-0 2026-02-20T18:09:15.766 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:175: rados_put_get_data: grep '\<2\>' 2026-02-20T18:09:15.766 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:09:15.766 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-85808-0 2026-02-20T18:09:15.766 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-85808-0 2026-02-20T18:09:15.767 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:09:15.994 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T18:09:15.994 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:09:15.994 INFO:tasks.workunit.client.0.trial195.stderr:0' 2026-02-20T18:09:15.994 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 0 2026-02-20T18:09:15.995 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:176: rados_put_get_data: ceph osd in 2 2026-02-20T18:09:16.241 INFO:tasks.workunit.client.0.trial195.stderr:osd.2 is already in. 2026-02-20T18:09:16.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:177: rados_put_get_data: activate_osd td/test-erasure-eio 2 2026-02-20T18:09:16.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-20T18:09:16.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-20T18:09:16.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-20T18:09:16.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-20T18:09:16.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-20T18:09:16.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:09:16.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:09:16.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:09:16.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:09:16.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-20T18:09:16.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-20T18:09:16.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-20T18:09:16.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-20T18:09:16.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:09:16.250 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-20T18:09:16.250 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:09:16.250 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:09:16.251 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:09:16.251 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:09:16.251 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:09:16.251 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:09:16.251 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:09:16.251 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:09:16.251 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:09:16.251 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:09:16.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:09:16.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:09:16.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:09:16.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:09:16.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-20T18:09:16.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-20T18:09:16.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-20T18:09:16.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-20T18:09:16.252 INFO:tasks.workunit.client.0.trial195.stdout:start osd.2 2026-02-20T18:09:16.252 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:09:16.252 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-20T18:09:16.253 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-20T18:09:16.253 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-20T18:09:16.253 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-20T18:09:16.253 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-20T18:09:16.269 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:16.268+0000 7f442f78c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:16.270 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:16.269+0000 7f442f78c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:16.271 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:16.270+0000 7f442f78c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:16.407 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:16.405+0000 7f442f78c900 -1 Falling back to public interface 2026-02-20T18:09:16.481 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-20T18:09:16.481 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:09:16.481 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-20T18:09:16.481 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:09:16.481 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:09:16.481 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:09:16.481 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:09:16.482 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:09:16.482 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:09:16.482 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:09:16.708 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:09:16.738 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:16.737+0000 7f442f78c900 -1 osd.2 35 log_to_monitors true 2026-02-20T18:09:17.710 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:09:17.710 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:09:17.710 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:09:17.710 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:09:17.710 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:09:17.710 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:09:17.938 INFO:tasks.workunit.client.0.trial195.stdout:osd.2 up in weight 1 up_from 44 up_thru 19 down_at 36 last_clean_interval [19,35) [v2:127.0.0.1:6818/3284350414,v1:127.0.0.1:6819/3284350414] [v2:127.0.0.1:6820/3284350414,v1:127.0.0.1:6821/3284350414] exists,up 6d767e96-3c95-4464-92bf-dc0ea1ab44fb 2026-02-20T18:09:17.938 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:09:17.938 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:09:17.938 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:09:17.938 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:178: rados_put_get_data: wait_for_clean 2026-02-20T18:09:17.938 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:09:17.938 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:09:17.938 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:09:17.938 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:09:17.939 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:09:17.939 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:09:17.939 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:09:17.939 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:09:17.939 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:09:18.026 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:09:18.027 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:09:18.027 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:09:18.027 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:09:18.027 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:09:18.027 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:09:18.253 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:09:18.253 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:09:18.253 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:09:18.253 INFO:tasks.workunit.client.0.trial195.stderr:3' 2026-02-20T18:09:18.253 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:09:18.253 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:09:18.253 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:09:18.317 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803784 2026-02-20T18:09:18.317 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803784 2026-02-20T18:09:18.317 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803784' 2026-02-20T18:09:18.317 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:09:18.317 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:09:18.383 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574856 2026-02-20T18:09:18.383 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574856 2026-02-20T18:09:18.383 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803784 1-55834574856' 2026-02-20T18:09:18.383 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:09:18.383 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:09:18.447 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=188978561027 2026-02-20T18:09:18.448 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 188978561027 2026-02-20T18:09:18.448 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803784 1-55834574856 2-188978561027' 2026-02-20T18:09:18.448 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:09:18.448 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:09:18.516 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964116998 2026-02-20T18:09:18.516 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964116998 2026-02-20T18:09:18.517 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803784 1-55834574856 2-188978561027 3-115964116998' 2026-02-20T18:09:18.517 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:09:18.517 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803784 2026-02-20T18:09:18.517 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:09:18.517 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:09:18.518 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803784 2026-02-20T18:09:18.518 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:09:18.518 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803784 2026-02-20T18:09:18.518 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-20T18:09:18.519 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.0 seq 25769803784 2026-02-20T18:09:18.519 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:09:18.746 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803782 -lt 25769803784 2026-02-20T18:09:18.746 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:09:19.747 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:09:19.748 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:09:19.973 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803782 -lt 25769803784 2026-02-20T18:09:19.973 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:09:20.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-20T18:09:20.974 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:09:21.205 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803784 -lt 25769803784 2026-02-20T18:09:21.205 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:09:21.205 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574856 2026-02-20T18:09:21.205 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:09:21.206 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:09:21.207 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-55834574856 2026-02-20T18:09:21.207 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:09:21.208 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574856 2026-02-20T18:09:21.208 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 55834574856' 2026-02-20T18:09:21.209 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.1 seq 55834574856 2026-02-20T18:09:21.209 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:09:21.435 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574857 -lt 55834574856 2026-02-20T18:09:21.436 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:09:21.436 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-188978561027 2026-02-20T18:09:21.436 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:09:21.437 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:09:21.437 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-188978561027 2026-02-20T18:09:21.438 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:09:21.439 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=188978561027 2026-02-20T18:09:21.439 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 188978561027' 2026-02-20T18:09:21.439 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.2 seq 188978561027 2026-02-20T18:09:21.439 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:09:21.668 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 188978561027 -lt 188978561027 2026-02-20T18:09:21.668 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:09:21.668 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964116998 2026-02-20T18:09:21.669 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:09:21.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:09:21.670 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-115964116998 2026-02-20T18:09:21.670 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:09:21.671 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=115964116998 2026-02-20T18:09:21.671 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 115964116998' 2026-02-20T18:09:21.671 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.3 seq 115964116998 2026-02-20T18:09:21.672 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:09:21.902 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 115964116999 -lt 115964116998 2026-02-20T18:09:21.902 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:09:21.903 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:09:21.903 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:09:22.217 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:09:22.217 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:09:22.218 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:09:22.218 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:09:22.218 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:09:22.218 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:09:22.218 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:09:22.218 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:09:22.443 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:09:22.443 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:09:22.444 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:09:22.444 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:09:22.756 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:09:22.756 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:09:22.756 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:09:22.756 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:360: TEST_rados_get_with_subreadall_eio_shard_0: delete_erasure_coded_pool pool-jerasure 2026-02-20T18:09:22.756 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-20T18:09:22.756 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-20T18:09:23.023 INFO:tasks.workunit.client.0.trial195.stderr:pool 'pool-jerasure' does not exist 2026-02-20T18:09:23.031 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-20T18:09:23.305 INFO:tasks.workunit.client.0.trial195.stderr:erasure-code-profile myprofile does not exist 2026-02-20T18:09:23.314 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-20T18:09:23.314 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-20T18:09:23.314 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T18:09:23.314 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-20T18:09:23.314 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:09:23.314 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:09:23.314 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:09:23.314 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:09:23.314 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:09:23.423 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:09:23.424 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T18:09:23.424 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T18:09:23.425 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T18:09:23.426 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T18:09:23.426 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T18:09:23.427 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T18:09:23.427 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:09:23.428 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T18:09:23.428 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T18:09:23.428 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:09:23.429 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T18:09:23.430 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T18:09:23.430 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-20T18:09:23.440 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T18:09:23.440 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:09:23.440 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:09:23.441 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.85808 2026-02-20T18:09:23.442 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T18:09:23.442 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T18:09:23.442 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-20T18:09:23.443 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-20T18:09:23.443 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-20T18:09:23.443 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-20T18:09:23.443 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-20T18:09:23.443 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T18:09:23.443 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-20T18:09:23.443 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:09:23.443 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:09:23.443 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:09:23.443 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:09:23.443 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:09:23.444 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:09:23.444 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T18:09:23.445 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T18:09:23.445 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T18:09:23.446 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T18:09:23.446 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T18:09:23.447 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T18:09:23.448 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:09:23.448 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T18:09:23.448 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T18:09:23.448 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:09:23.449 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T18:09:23.450 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T18:09:23.450 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-20T18:09:23.452 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T18:09:23.452 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:09:23.452 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:09:23.452 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.85808 2026-02-20T18:09:23.454 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T18:09:23.454 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T18:09:23.454 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-20T18:09:23.454 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-20T18:09:23.454 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:09:23.454 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:09:23.455 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.85808 2026-02-20T18:09:23.456 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-20T18:09:23.456 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-20T18:09:23.456 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-20T18:09:23.456 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-20T18:09:23.456 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-20T18:09:23.456 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-20T18:09:23.456 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-20T18:09:23.456 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-20T18:09:23.457 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-20T18:09:23.457 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-20T18:09:23.457 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-20T18:09:23.477 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-20T18:09:23.477 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:09:23.477 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:09:23.478 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:09:23.478 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:09:23.478 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:09:23.478 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:09:23.478 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-20T18:09:23.505 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-20T18:09:23.506 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-20T18:09:23.507 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T18:09:23.507 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T18:09:23.507 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-20T18:09:23.507 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-20T18:09:23.507 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T18:09:23.507 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:09:23.507 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:09:23.509 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:09:23.509 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:09:23.509 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:09:23.509 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:09:23.509 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T18:09:23.509 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.85808/ceph-mon.a.asok config get fsid 2026-02-20T18:09:23.553 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-20T18:09:23.553 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-20T18:09:23.553 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-20T18:09:23.553 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-20T18:09:23.553 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-20T18:09:23.553 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-20T18:09:23.554 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:09:23.554 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:09:23.554 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:09:23.554 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:09:23.554 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:09:23.554 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:09:23.555 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-20T18:09:23.555 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.85808/ceph-mon.a.asok config get mon_host 2026-02-20T18:09:23.598 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-20T18:09:23.598 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-20T18:09:23.598 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-20T18:09:23.598 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-20T18:09:23.598 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-20T18:09:23.598 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-20T18:09:23.599 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-20T18:09:23.696 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-20T18:09:23.696 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:09:23.696 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:09:23.696 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:09:23.696 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:09:23.696 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:09:23.697 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:09:23.697 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T18:09:23.698 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-20T18:09:23.716 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-20T18:09:23.716 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-20T18:09:23.813 INFO:tasks.workunit.client.0.trial195.stderr:pool 'rbd' already exists 2026-02-20T18:09:23.819 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-20T18:09:24.822 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-20T18:09:24.822 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-20T18:09:24.823 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-20T18:09:24.823 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:09:24.823 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:09:24.823 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:09:24.823 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-mon.a.asok 2026-02-20T18:09:24.823 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-20T18:09:24.823 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-mon.a.asok log flush 2026-02-20T18:09:24.863 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T18:09:24.867 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-20T18:09:24.868 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T18:09:23.496+0000 7f4d83ebad80 0 load: jerasure load: lrc 2026-02-20T18:09:24.869 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_with_subreadall_eio_shard_1 td/test-erasure-eio 2026-02-20T18:09:24.869 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:364: TEST_rados_get_with_subreadall_eio_shard_1: local dir=td/test-erasure-eio 2026-02-20T18:09:24.869 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:365: TEST_rados_get_with_subreadall_eio_shard_1: local shard_id=1 2026-02-20T18:09:24.869 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:367: TEST_rados_get_with_subreadall_eio_shard_1: setup_osds 4 2026-02-20T18:09:24.869 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-20T18:09:24.869 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-20T18:09:24.869 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-20T18:09:24.874 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-20T18:09:24.874 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:09:24.874 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-20T18:09:24.874 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:09:24.874 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:09:24.874 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-20T18:09:24.874 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:09:24.874 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-20T18:09:24.875 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:09:24.875 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:09:24.875 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:09:24.875 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:09:24.875 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-20T18:09:24.875 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-20T18:09:24.875 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:09:24.875 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:09:24.875 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:09:24.875 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:09:24.875 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:09:24.875 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:09:24.876 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:09:24.876 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:09:24.876 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:09:24.876 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:09:24.876 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:09:24.876 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:09:24.876 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:09:24.876 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:09:24.876 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:09:24.876 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:09:24.876 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:09:24.876 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:09:24.877 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:09:24.877 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:09:24.877 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:09:24.877 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:09:24.877 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-20T18:09:24.877 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:09:24.877 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b0b82814-36cb-4c66-9cb3-123122414ccf 2026-02-20T18:09:24.877 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 b0b82814-36cb-4c66-9cb3-123122414ccf' 2026-02-20T18:09:24.877 INFO:tasks.workunit.client.0.trial195.stdout:add osd0 b0b82814-36cb-4c66-9cb3-123122414ccf 2026-02-20T18:09:24.878 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:09:24.886 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBUo5hpbLS/NBAA/USm6Y6+trkO1pFstGCnlw== 2026-02-20T18:09:24.886 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBUo5hpbLS/NBAA/USm6Y6+trkO1pFstGCnlw=="}' 2026-02-20T18:09:24.886 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b0b82814-36cb-4c66-9cb3-123122414ccf -i td/test-erasure-eio/0/new.json 2026-02-20T18:09:24.991 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:09:24.997 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-20T18:09:24.998 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBUo5hpbLS/NBAA/USm6Y6+trkO1pFstGCnlw== --osd-uuid b0b82814-36cb-4c66-9cb3-123122414ccf 2026-02-20T18:09:25.016 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:25.014+0000 7f77eb9dc900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:25.017 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:25.016+0000 7f77eb9dc900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:25.018 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:25.017+0000 7f77eb9dc900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:25.019 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:25.017+0000 7f77eb9dc900 -1 bdev(0x558463180c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-20T18:09:25.019 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:25.017+0000 7f77eb9dc900 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-20T18:09:25.466 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-20T18:09:25.466 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:09:25.466 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-20T18:09:25.467 INFO:tasks.workunit.client.0.trial195.stdout:adding osd0 key to auth repository 2026-02-20T18:09:25.467 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:09:25.571 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-20T18:09:25.572 INFO:tasks.workunit.client.0.trial195.stdout:start osd.0 2026-02-20T18:09:25.573 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:09:25.573 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:09:25.573 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:09:25.573 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:09:25.590 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:25.588+0000 7fbe7476b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:25.591 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:25.589+0000 7fbe7476b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:25.592 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:25.590+0000 7fbe7476b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:25.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-20T18:09:25.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:09:25.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-20T18:09:25.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:09:25.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:09:25.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:09:25.669 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:09:25.670 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:09:25.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:09:25.670 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T18:09:25.770 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:09:25.782 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:25.780+0000 7fbe7476b900 -1 Falling back to public interface 2026-02-20T18:09:26.029 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:26.027+0000 7fbe7476b900 -1 osd.0 0 log_to_monitors true 2026-02-20T18:09:26.772 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:09:26.772 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:09:26.772 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:09:26.773 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:09:26.773 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:09:26.773 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-20T18:09:26.998 INFO:tasks.workunit.client.0.trial195.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1783704488,v1:127.0.0.1:6803/1783704488] [v2:127.0.0.1:6804/1783704488,v1:127.0.0.1:6805/1783704488] exists,up b0b82814-36cb-4c66-9cb3-123122414ccf 2026-02-20T18:09:27.002 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:09:27.002 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:09:27.002 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:09:27.003 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:09:27.003 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-20T18:09:27.003 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:09:27.003 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:09:27.003 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-20T18:09:27.003 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:09:27.003 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-20T18:09:27.003 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:09:27.003 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:09:27.003 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:09:27.003 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:09:27.003 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-20T18:09:27.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-20T18:09:27.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:09:27.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:09:27.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:09:27.004 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:09:27.004 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:09:27.004 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:09:27.004 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:09:27.004 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:09:27.004 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:09:27.004 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:09:27.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:09:27.005 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:09:27.005 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:09:27.005 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:09:27.005 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:09:27.005 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:09:27.005 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:09:27.005 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:09:27.005 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:09:27.005 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:09:27.005 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:09:27.005 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:09:27.005 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-20T18:09:27.006 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:09:27.006 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=952a700a-9bc0-4258-b74e-a7e31aa8aba2 2026-02-20T18:09:27.006 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 952a700a-9bc0-4258-b74e-a7e31aa8aba2' 2026-02-20T18:09:27.006 INFO:tasks.workunit.client.0.trial195.stdout:add osd1 952a700a-9bc0-4258-b74e-a7e31aa8aba2 2026-02-20T18:09:27.006 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:09:27.014 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBXo5hp1v/IABAAbVw8UbHWC79TdwI0d3/cPg== 2026-02-20T18:09:27.015 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBXo5hp1v/IABAAbVw8UbHWC79TdwI0d3/cPg=="}' 2026-02-20T18:09:27.015 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 952a700a-9bc0-4258-b74e-a7e31aa8aba2 -i td/test-erasure-eio/1/new.json 2026-02-20T18:09:27.244 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:09:27.253 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-20T18:09:27.254 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBXo5hp1v/IABAAbVw8UbHWC79TdwI0d3/cPg== --osd-uuid 952a700a-9bc0-4258-b74e-a7e31aa8aba2 2026-02-20T18:09:27.271 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:27.269+0000 7f32d1eec900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:27.273 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:27.271+0000 7f32d1eec900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:27.274 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:27.272+0000 7f32d1eec900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:27.274 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:27.272+0000 7f32d1eec900 -1 bdev(0x555a26766c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-20T18:09:27.274 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:27.272+0000 7f32d1eec900 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-20T18:09:27.612 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-20T18:09:27.612 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:09:27.613 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-20T18:09:27.613 INFO:tasks.workunit.client.0.trial195.stdout:adding osd1 key to auth repository 2026-02-20T18:09:27.613 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:09:27.928 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-20T18:09:27.929 INFO:tasks.workunit.client.0.trial195.stdout:start osd.1 2026-02-20T18:09:27.929 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:09:27.929 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:09:27.929 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:09:27.929 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:09:27.946 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:27.944+0000 7fb2a6814900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:27.948 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:27.946+0000 7fb2a6814900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:27.948 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:27.946+0000 7fb2a6814900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:28.090 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:28.088+0000 7fb2a6814900 -1 Falling back to public interface 2026-02-20T18:09:28.156 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-20T18:09:28.156 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:09:28.156 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-20T18:09:28.156 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:09:28.156 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:09:28.156 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:09:28.156 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:09:28.156 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:09:28.156 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:09:28.156 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:09:28.380 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:09:28.411 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:28.409+0000 7fb2a6814900 -1 osd.1 0 log_to_monitors true 2026-02-20T18:09:29.381 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:09:29.381 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:09:29.382 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:09:29.382 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:09:29.383 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:09:29.383 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-20T18:09:29.609 INFO:tasks.workunit.client.0.trial195.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3115651694,v1:127.0.0.1:6811/3115651694] [v2:127.0.0.1:6812/3115651694,v1:127.0.0.1:6813/3115651694] exists,up 952a700a-9bc0-4258-b74e-a7e31aa8aba2 2026-02-20T18:09:29.610 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:09:29.610 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:09:29.610 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:09:29.610 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:09:29.610 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-20T18:09:29.610 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:09:29.610 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:09:29.610 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-20T18:09:29.610 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:09:29.610 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-20T18:09:29.610 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:09:29.610 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:09:29.610 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:09:29.610 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:09:29.610 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-20T18:09:29.610 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-20T18:09:29.610 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:09:29.610 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:09:29.610 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:09:29.610 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:09:29.610 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:09:29.610 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:09:29.610 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:09:29.610 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:09:29.610 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:09:29.611 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:09:29.611 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:09:29.611 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:09:29.611 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:09:29.611 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:09:29.611 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:09:29.611 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:09:29.611 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:09:29.611 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:09:29.611 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:09:29.611 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:09:29.611 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:09:29.611 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:09:29.611 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-20T18:09:29.612 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:09:29.613 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f9d66997-76dc-4734-8e80-5c5558bf6412 2026-02-20T18:09:29.613 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 f9d66997-76dc-4734-8e80-5c5558bf6412' 2026-02-20T18:09:29.613 INFO:tasks.workunit.client.0.trial195.stdout:add osd2 f9d66997-76dc-4734-8e80-5c5558bf6412 2026-02-20T18:09:29.613 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:09:29.625 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBZo5hpxJ4zJRAAjeykp04P3dWSP3zaKF018w== 2026-02-20T18:09:29.625 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBZo5hpxJ4zJRAAjeykp04P3dWSP3zaKF018w=="}' 2026-02-20T18:09:29.625 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f9d66997-76dc-4734-8e80-5c5558bf6412 -i td/test-erasure-eio/2/new.json 2026-02-20T18:09:29.854 INFO:tasks.workunit.client.0.trial195.stdout:2 2026-02-20T18:09:29.862 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-20T18:09:29.863 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBZo5hpxJ4zJRAAjeykp04P3dWSP3zaKF018w== --osd-uuid f9d66997-76dc-4734-8e80-5c5558bf6412 2026-02-20T18:09:29.882 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:29.880+0000 7f2e271c8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:29.883 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:29.881+0000 7f2e271c8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:29.884 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:29.882+0000 7f2e271c8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:29.885 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:29.883+0000 7f2e271c8900 -1 bdev(0x55a9e52bac00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-20T18:09:29.885 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:29.883+0000 7f2e271c8900 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-20T18:09:30.216 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-20T18:09:30.216 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:09:30.217 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-20T18:09:30.217 INFO:tasks.workunit.client.0.trial195.stdout:adding osd2 key to auth repository 2026-02-20T18:09:30.217 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:09:30.533 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-20T18:09:30.534 INFO:tasks.workunit.client.0.trial195.stdout:start osd.2 2026-02-20T18:09:30.534 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:09:30.535 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:09:30.535 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:09:30.535 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:09:30.550 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:30.548+0000 7fd579b46900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:30.551 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:30.550+0000 7fd579b46900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:30.552 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:30.550+0000 7fd579b46900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:30.682 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:30.680+0000 7fd579b46900 -1 Falling back to public interface 2026-02-20T18:09:30.758 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-20T18:09:30.758 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:09:30.758 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-20T18:09:30.758 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:09:30.758 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:09:30.758 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:09:30.759 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:09:30.759 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:09:30.759 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:09:30.760 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:09:30.983 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:09:31.007 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:31.005+0000 7fd579b46900 -1 osd.2 0 log_to_monitors true 2026-02-20T18:09:31.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:09:31.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:09:31.985 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:09:31.986 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:09:31.986 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:09:31.986 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:09:32.209 INFO:tasks.workunit.client.0.trial195.stdout:osd.2 up in weight 1 up_from 17 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1871795157,v1:127.0.0.1:6819/1871795157] [v2:127.0.0.1:6820/1871795157,v1:127.0.0.1:6821/1871795157] exists,up f9d66997-76dc-4734-8e80-5c5558bf6412 2026-02-20T18:09:32.218 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:09:32.218 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:09:32.218 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:09:32.218 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-20T18:09:32.219 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-20T18:09:32.219 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-20T18:09:32.219 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-20T18:09:32.219 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-20T18:09:32.219 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-20T18:09:32.219 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-20T18:09:32.220 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:09:32.220 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:09:32.220 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:09:32.220 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:09:32.220 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-20T18:09:32.220 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-20T18:09:32.220 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-20T18:09:32.220 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-20T18:09:32.220 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:09:32.220 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-20T18:09:32.220 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:09:32.221 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:09:32.221 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:09:32.221 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:09:32.221 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:09:32.221 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:09:32.221 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:09:32.221 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:09:32.221 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-20T18:09:32.221 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-20T18:09:32.221 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:09:32.221 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:09:32.221 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:09:32.222 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:09:32.222 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:09:32.222 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:09:32.222 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-20T18:09:32.222 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-20T18:09:32.222 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-20T18:09:32.222 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-20T18:09:32.222 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e9e2c500-8a1a-4fbb-b335-26616531f35e 2026-02-20T18:09:32.222 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 e9e2c500-8a1a-4fbb-b335-26616531f35e' 2026-02-20T18:09:32.223 INFO:tasks.workunit.client.0.trial195.stdout:add osd3 e9e2c500-8a1a-4fbb-b335-26616531f35e 2026-02-20T18:09:32.223 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-20T18:09:32.224 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBco5hp1ZVKDRAAQKun4C/r37E2ftJlIf4plQ== 2026-02-20T18:09:32.225 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBco5hp1ZVKDRAAQKun4C/r37E2ftJlIf4plQ=="}' 2026-02-20T18:09:32.225 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e9e2c500-8a1a-4fbb-b335-26616531f35e -i td/test-erasure-eio/3/new.json 2026-02-20T18:09:32.453 INFO:tasks.workunit.client.0.trial195.stdout:3 2026-02-20T18:09:32.461 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-20T18:09:32.462 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBco5hp1ZVKDRAAQKun4C/r37E2ftJlIf4plQ== --osd-uuid e9e2c500-8a1a-4fbb-b335-26616531f35e 2026-02-20T18:09:32.479 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:32.477+0000 7f70b8dc1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:32.481 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:32.479+0000 7f70b8dc1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:32.481 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:32.480+0000 7f70b8dc1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:32.482 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:32.480+0000 7f70b8dc1900 -1 bdev(0x55d7a754cc00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-20T18:09:32.482 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:32.480+0000 7f70b8dc1900 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-20T18:09:32.797 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-20T18:09:32.797 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-20T18:09:32.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-20T18:09:32.798 INFO:tasks.workunit.client.0.trial195.stdout:adding osd3 key to auth repository 2026-02-20T18:09:32.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-20T18:09:33.116 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-20T18:09:33.117 INFO:tasks.workunit.client.0.trial195.stdout:start osd.3 2026-02-20T18:09:33.117 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:09:33.117 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-20T18:09:33.117 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-20T18:09:33.117 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-20T18:09:33.134 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:33.132+0000 7f7c56379900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:33.136 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:33.134+0000 7f7c56379900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:33.137 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:33.135+0000 7f7c56379900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:33.283 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:33.280+0000 7f7c56379900 -1 Falling back to public interface 2026-02-20T18:09:33.346 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-20T18:09:33.346 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:09:33.346 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-20T18:09:33.347 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:09:33.347 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:09:33.347 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:09:33.347 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:09:33.347 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:09:33.347 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:09:33.347 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T18:09:33.573 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:09:33.616 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:33.614+0000 7f7c56379900 -1 osd.3 0 log_to_monitors true 2026-02-20T18:09:34.575 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:09:34.575 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:09:34.575 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:09:34.575 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:09:34.576 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:09:34.576 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-20T18:09:34.797 INFO:tasks.workunit.client.0.trial195.stdout:osd.3 up in weight 1 up_from 25 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/561602193,v1:127.0.0.1:6827/561602193] [v2:127.0.0.1:6828/561602193,v1:127.0.0.1:6829/561602193] exists,up e9e2c500-8a1a-4fbb-b335-26616531f35e 2026-02-20T18:09:34.797 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:09:34.797 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:09:34.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:09:34.798 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-20T18:09:34.798 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-20T18:09:34.798 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-20T18:09:34.798 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:09:34.798 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:09:34.798 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:09:34.798 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.0.asok 2026-02-20T18:09:34.799 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-20T18:09:34.799 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-osd.0.asok log flush 2026-02-20T18:09:34.845 INFO:tasks.workunit.client.0.trial195.stdout:{} 2026-02-20T18:09:34.848 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-20T18:09:34.851 INFO:tasks.workunit.client.0.trial195.stdout:2026-02-20T18:09:25.783+0000 7fbe7476b900 0 load: jerasure load: lrc 2026-02-20T18:09:34.852 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:369: TEST_rados_get_with_subreadall_eio_shard_1: local poolname=pool-jerasure 2026-02-20T18:09:34.852 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:370: TEST_rados_get_with_subreadall_eio_shard_1: create_erasure_coded_pool pool-jerasure 2 1 2026-02-20T18:09:34.852 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-20T18:09:34.852 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-20T18:09:34.852 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-20T18:09:34.852 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-20T18:09:34.852 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-20T18:09:34.853 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-20T18:09:34.853 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-20T18:09:35.134 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-20T18:09:35.134 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-20T18:09:35.506 INFO:tasks.workunit.client.0.trial195.stderr:pool 'pool-jerasure' already exists 2026-02-20T18:09:35.514 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-20T18:09:36.515 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-20T18:09:36.516 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:09:36.516 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:09:36.516 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:09:36.517 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:09:36.517 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:09:36.517 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:09:36.517 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:09:36.517 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:09:36.518 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:09:36.604 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:09:36.604 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:09:36.604 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:09:36.604 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:09:36.604 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:09:36.604 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:09:36.827 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:09:36.828 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:09:36.828 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:09:36.828 INFO:tasks.workunit.client.0.trial195.stderr:3' 2026-02-20T18:09:36.828 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:09:36.828 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:09:36.828 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:09:36.893 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803781 2026-02-20T18:09:36.894 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803781 2026-02-20T18:09:36.894 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781' 2026-02-20T18:09:36.894 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:09:36.894 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:09:36.958 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607556 2026-02-20T18:09:36.958 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607556 2026-02-20T18:09:36.958 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781 1-51539607556' 2026-02-20T18:09:36.958 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:09:36.959 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:09:37.025 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444035 2026-02-20T18:09:37.025 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444035 2026-02-20T18:09:37.025 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781 1-51539607556 2-73014444035' 2026-02-20T18:09:37.025 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:09:37.025 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:09:37.091 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182403 2026-02-20T18:09:37.091 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182403 2026-02-20T18:09:37.092 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781 1-51539607556 2-73014444035 3-107374182403' 2026-02-20T18:09:37.092 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:09:37.092 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803781 2026-02-20T18:09:37.092 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:09:37.092 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:09:37.092 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803781 2026-02-20T18:09:37.092 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:09:37.092 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803781 2026-02-20T18:09:37.092 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-20T18:09:37.093 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.0 seq 25769803781 2026-02-20T18:09:37.093 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:09:37.320 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803779 -lt 25769803781 2026-02-20T18:09:37.320 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:09:38.321 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:09:38.322 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:09:38.545 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803782 -lt 25769803781 2026-02-20T18:09:38.546 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:09:38.546 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607556 2026-02-20T18:09:38.546 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:09:38.547 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:09:38.547 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-51539607556 2026-02-20T18:09:38.548 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:09:38.549 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=51539607556 2026-02-20T18:09:38.549 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 51539607556' 2026-02-20T18:09:38.549 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.1 seq 51539607556 2026-02-20T18:09:38.549 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:09:38.774 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 51539607556 -lt 51539607556 2026-02-20T18:09:38.774 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:09:38.775 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444035 2026-02-20T18:09:38.775 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:09:38.775 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:09:38.776 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-73014444035 2026-02-20T18:09:38.776 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:09:38.777 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=73014444035 2026-02-20T18:09:38.777 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.2 seq 73014444035 2026-02-20T18:09:38.778 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 73014444035' 2026-02-20T18:09:38.778 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:09:39.001 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 73014444036 -lt 73014444035 2026-02-20T18:09:39.001 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:09:39.003 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182403 2026-02-20T18:09:39.003 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:09:39.003 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:09:39.003 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-107374182403 2026-02-20T18:09:39.003 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:09:39.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=107374182403 2026-02-20T18:09:39.004 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.3 seq 107374182403 2026-02-20T18:09:39.004 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 107374182403' 2026-02-20T18:09:39.005 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:09:39.232 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 107374182403 -lt 107374182403 2026-02-20T18:09:39.233 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:09:39.233 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:09:39.233 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:09:39.546 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:09:39.546 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:09:39.546 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:09:39.547 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:09:39.547 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:09:39.547 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:09:39.547 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:09:39.547 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:09:39.768 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:09:39.768 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:09:39.769 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:09:39.769 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:09:40.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:09:40.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:09:40.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:09:40.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:372: TEST_rados_get_with_subreadall_eio_shard_1: rados_put_get_data eio td/test-erasure-eio 1 recovery 2026-02-20T18:09:40.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=eio 2026-02-20T18:09:40.081 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-20T18:09:40.082 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-20T18:09:40.082 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-20T18:09:40.082 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=1 2026-02-20T18:09:40.082 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-20T18:09:40.082 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg=recovery 2026-02-20T18:09:40.082 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-20T18:09:40.082 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-eio-85808-1 2026-02-20T18:09:40.082 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-eio-85808-1 2026-02-20T18:09:40.082 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-20T18:09:40.082 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-20T18:09:40.082 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-eio-85808-1 2026-02-20T18:09:40.082 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:09:40.083 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-20T18:09:40.083 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:09:40.083 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-20T18:09:40.083 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:09:40.083 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-20T18:09:40.083 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-20T18:09:40.083 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-20T18:09:40.083 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-eio-85808-1 td/test-erasure-eio/ORIGINAL 2026-02-20T18:09:40.101 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-85808-1 td/test-erasure-eio 1 2026-02-20T18:09:40.101 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=ec 2026-02-20T18:09:40.102 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-20T18:09:40.102 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=data 2026-02-20T18:09:40.102 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-20T18:09:40.102 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=pool-jerasure 2026-02-20T18:09:40.102 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-20T18:09:40.102 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=obj-eio-85808-1 2026-02-20T18:09:40.102 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-20T18:09:40.102 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/test-erasure-eio 2026-02-20T18:09:40.102 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-20T18:09:40.102 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=1 2026-02-20T18:09:40.103 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-20T18:09:40.103 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds pool-jerasure obj-eio-85808-1 2026-02-20T18:09:40.103 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:09:40.103 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-85808-1 2026-02-20T18:09:40.103 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-85808-1 2026-02-20T18:09:40.103 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:09:40.330 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T18:09:40.330 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:09:40.330 INFO:tasks.workunit.client.0.trial195.stderr:2' 2026-02-20T18:09:40.330 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-20T18:09:40.331 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('3' '1' '2') 2026-02-20T18:09:40.331 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-20T18:09:40.331 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=1 2026-02-20T18:09:40.331 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' ec '!=' ec ']' 2026-02-20T18:09:40.331 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/test-erasure-eio/1/type 2026-02-20T18:09:40.331 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-20T18:09:40.332 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-20T18:09:40.332 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-20T18:09:40.332 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-20T18:09:40.332 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-20T18:09:40.332 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-20T18:09:40.332 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-20T18:09:40.332 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-20T18:09:40.332 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-20T18:09:40.332 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-20T18:09:40.333 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:09:40.333 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:09:40.333 INFO:tasks.workunit.client.0.trial195.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:09:40.333 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.1.asok 2026-02-20T18:09:40.333 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.85808/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-20T18:09:40.386 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-20T18:09:40.386 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-20T18:09:40.386 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-20T18:09:40.387 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.1 2026-02-20T18:09:40.387 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-20T18:09:40.387 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-20T18:09:40.387 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-20T18:09:40.387 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:09:40.387 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:09:40.387 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85808/ceph-osd.1.asok 2026-02-20T18:09:40.387 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-20T18:09:40.387 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.85808/ceph-osd.1.asok injectdataerr pool-jerasure obj-eio-85808-1 1 2026-02-20T18:09:40.437 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-85808-1 2026-02-20T18:09:40.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-20T18:09:40.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-20T18:09:40.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-85808-1 2026-02-20T18:09:40.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-20T18:09:40.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-20T18:09:40.438 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-85808-1 td/test-erasure-eio/COPY 2026-02-20T18:09:40.456 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-20T18:09:40.457 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-20T18:09:40.458 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' recovery = recovery ']' 2026-02-20T18:09:40.458 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: get_osds pool-jerasure obj-eio-85808-1 2026-02-20T18:09:40.458 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:09:40.458 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-85808-1 2026-02-20T18:09:40.459 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-85808-1 2026-02-20T18:09:40.459 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:09:40.687 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T18:09:40.687 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:09:40.687 INFO:tasks.workunit.client.0.trial195.stderr:2' 2026-02-20T18:09:40.687 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-20T18:09:40.687 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: initial_osds=('3' '1' '2') 2026-02-20T18:09:40.687 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: local -a initial_osds 2026-02-20T18:09:40.688 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:171: rados_put_get_data: local last_osd=2 2026-02-20T18:09:40.688 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:173: rados_put_get_data: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-20T18:09:40.688 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:09:40.688 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:09:40.688 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:09:40.688 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:09:40.688 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:09:40.994 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:09:40.994 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:174: rados_put_get_data: ceph osd out 2 2026-02-20T18:09:41.269 INFO:tasks.workunit.client.0.trial195.stderr:osd.2 is already out. 2026-02-20T18:09:41.277 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:175: rados_put_get_data: get_osds pool-jerasure obj-eio-85808-1 2026-02-20T18:09:41.278 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:175: rados_put_get_data: grep '\<2\>' 2026-02-20T18:09:41.278 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-20T18:09:41.278 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-85808-1 2026-02-20T18:09:41.278 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-85808-1 2026-02-20T18:09:41.278 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-20T18:09:41.504 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-20T18:09:41.505 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:09:41.505 INFO:tasks.workunit.client.0.trial195.stderr:0' 2026-02-20T18:09:41.505 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 0 2026-02-20T18:09:41.505 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:176: rados_put_get_data: ceph osd in 2 2026-02-20T18:09:41.785 INFO:tasks.workunit.client.0.trial195.stderr:osd.2 is already in. 2026-02-20T18:09:41.797 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:177: rados_put_get_data: activate_osd td/test-erasure-eio 2 2026-02-20T18:09:41.797 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-20T18:09:41.797 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-20T18:09:41.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-20T18:09:41.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-20T18:09:41.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-20T18:09:41.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-20T18:09:41.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-20T18:09:41.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-20T18:09:41.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-20T18:09:41.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-20T18:09:41.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-20T18:09:41.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-20T18:09:41.798 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-20T18:09:41.799 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-20T18:09:41.799 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-20T18:09:41.799 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-20T18:09:41.799 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-20T18:09:41.799 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-20T18:09:41.799 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:09:41.799 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:09:41.799 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:09:41.799 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' 2026-02-20T18:09:41.799 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-20T18:09:41.799 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-20T18:09:41.799 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-20T18:09:41.800 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-20T18:09:41.800 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-20T18:09:41.800 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-20T18:09:41.800 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-20T18:09:41.800 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-20T18:09:41.800 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-20T18:09:41.800 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-20T18:09:41.800 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-20T18:09:41.800 INFO:tasks.workunit.client.0.trial195.stdout:start osd.2 2026-02-20T18:09:41.801 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=0f2629cd-f9f3-4f2d-a30b-b891e04eae71 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.85808/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-20T18:09:41.801 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-20T18:09:41.802 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-20T18:09:41.802 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-20T18:09:41.802 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-20T18:09:41.802 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-20T18:09:41.815 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:41.813+0000 7f7716227900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:41.816 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:41.814+0000 7f7716227900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:41.817 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:41.815+0000 7f7716227900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-20T18:09:41.977 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:41.975+0000 7f7716227900 -1 Falling back to public interface 2026-02-20T18:09:42.024 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-20T18:09:42.024 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-20T18:09:42.024 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-20T18:09:42.024 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-20T18:09:42.025 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-20T18:09:42.025 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:09:42.025 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-20T18:09:42.025 INFO:tasks.workunit.client.0.trial195.stdout:0 2026-02-20T18:09:42.025 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:09:42.025 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:09:42.247 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-20T18:09:42.343 INFO:tasks.workunit.client.0.trial195.stderr:2026-02-20T18:09:42.341+0000 7f7716227900 -1 osd.2 32 log_to_monitors true 2026-02-20T18:09:43.249 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-20T18:09:43.249 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-20T18:09:43.249 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-20T18:09:43.249 INFO:tasks.workunit.client.0.trial195.stdout:1 2026-02-20T18:09:43.249 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-20T18:09:43.250 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-20T18:09:43.476 INFO:tasks.workunit.client.0.trial195.stdout:osd.2 up in weight 1 up_from 41 up_thru 17 down_at 33 last_clean_interval [17,32) [v2:127.0.0.1:6818/64344165,v1:127.0.0.1:6819/64344165] [v2:127.0.0.1:6820/64344165,v1:127.0.0.1:6821/64344165] exists,up f9d66997-76dc-4734-8e80-5c5558bf6412 2026-02-20T18:09:43.477 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-20T18:09:43.477 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-20T18:09:43.477 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-20T18:09:43.477 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:178: rados_put_get_data: wait_for_clean 2026-02-20T18:09:43.477 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-20T18:09:43.477 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-20T18:09:43.477 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-20T18:09:43.477 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-20T18:09:43.477 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-20T18:09:43.478 INFO:tasks.workunit.client.0.trial195.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-20T18:09:43.478 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-20T18:09:43.478 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-20T18:09:43.478 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-20T18:09:43.563 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-20T18:09:43.563 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-20T18:09:43.563 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-20T18:09:43.563 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-20T18:09:43.563 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-20T18:09:43.564 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-20T18:09:43.791 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-20T18:09:43.791 INFO:tasks.workunit.client.0.trial195.stderr:1 2026-02-20T18:09:43.791 INFO:tasks.workunit.client.0.trial195.stderr:2 2026-02-20T18:09:43.791 INFO:tasks.workunit.client.0.trial195.stderr:3' 2026-02-20T18:09:43.792 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-20T18:09:43.792 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:09:43.792 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-20T18:09:43.855 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803785 2026-02-20T18:09:43.855 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803785 2026-02-20T18:09:43.855 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803785' 2026-02-20T18:09:43.855 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:09:43.855 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-20T18:09:43.919 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607559 2026-02-20T18:09:43.919 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607559 2026-02-20T18:09:43.919 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803785 1-51539607559' 2026-02-20T18:09:43.919 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:09:43.919 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-20T18:09:43.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=176093659139 2026-02-20T18:09:43.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 176093659139 2026-02-20T18:09:43.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803785 1-51539607559 2-176093659139' 2026-02-20T18:09:43.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-20T18:09:43.985 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-20T18:09:44.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182406 2026-02-20T18:09:44.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182406 2026-02-20T18:09:44.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803785 1-51539607559 2-176093659139 3-107374182406' 2026-02-20T18:09:44.050 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:09:44.051 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803785 2026-02-20T18:09:44.051 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:09:44.051 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-20T18:09:44.052 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803785 2026-02-20T18:09:44.052 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:09:44.054 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803785 2026-02-20T18:09:44.054 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803785' 2026-02-20T18:09:44.054 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.0 seq 25769803785 2026-02-20T18:09:44.054 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:09:44.282 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803783 -lt 25769803785 2026-02-20T18:09:44.282 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:09:45.283 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-20T18:09:45.284 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:09:45.514 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803783 -lt 25769803785 2026-02-20T18:09:45.514 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-20T18:09:46.515 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-20T18:09:46.515 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-20T18:09:46.743 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803785 -lt 25769803785 2026-02-20T18:09:46.743 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:09:46.744 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607559 2026-02-20T18:09:46.744 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:09:46.744 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-20T18:09:46.745 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-51539607559 2026-02-20T18:09:46.745 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:09:46.746 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=51539607559 2026-02-20T18:09:46.746 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 51539607559' 2026-02-20T18:09:46.747 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.1 seq 51539607559 2026-02-20T18:09:46.747 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-20T18:09:46.976 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 51539607560 -lt 51539607559 2026-02-20T18:09:46.976 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:09:46.976 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-176093659139 2026-02-20T18:09:46.977 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:09:46.977 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-20T18:09:46.978 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-176093659139 2026-02-20T18:09:46.978 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:09:46.979 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=176093659139 2026-02-20T18:09:46.979 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 176093659139' 2026-02-20T18:09:46.979 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.2 seq 176093659139 2026-02-20T18:09:46.980 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-20T18:09:47.205 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 176093659139 -lt 176093659139 2026-02-20T18:09:47.205 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-20T18:09:47.205 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182406 2026-02-20T18:09:47.205 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-20T18:09:47.206 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-20T18:09:47.207 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-107374182406 2026-02-20T18:09:47.207 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-20T18:09:47.208 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=107374182406 2026-02-20T18:09:47.208 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 107374182406' 2026-02-20T18:09:47.208 INFO:tasks.workunit.client.0.trial195.stdout:waiting osd.3 seq 107374182406 2026-02-20T18:09:47.208 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-20T18:09:47.440 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 107374182407 -lt 107374182406 2026-02-20T18:09:47.441 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-20T18:09:47.441 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:09:47.441 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:09:47.749 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-20T18:09:47.749 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-20T18:09:47.750 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-20T18:09:47.750 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-20T18:09:47.750 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-20T18:09:47.750 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-20T18:09:47.750 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-20T18:09:47.750 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-20T18:09:47.974 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-20T18:09:47.974 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-20T18:09:47.975 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-20T18:09:47.975 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-20T18:09:48.291 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-20T18:09:48.291 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-20T18:09:48.292 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-20T18:09:48.292 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:374: TEST_rados_get_with_subreadall_eio_shard_1: delete_erasure_coded_pool pool-jerasure 2026-02-20T18:09:48.292 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-20T18:09:48.292 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-20T18:09:48.561 INFO:tasks.workunit.client.0.trial195.stderr:pool 'pool-jerasure' does not exist 2026-02-20T18:09:48.570 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-20T18:09:48.847 INFO:tasks.workunit.client.0.trial195.stderr:erasure-code-profile myprofile does not exist 2026-02-20T18:09:48.855 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-20T18:09:48.855 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-20T18:09:48.855 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-20T18:09:48.855 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-20T18:09:48.855 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:09:48.855 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:09:48.856 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:09:48.856 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:09:48.856 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:09:48.966 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:09:48.966 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T18:09:48.967 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T18:09:48.968 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T18:09:48.969 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T18:09:48.970 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T18:09:48.970 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T18:09:48.970 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:09:48.971 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T18:09:48.971 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T18:09:48.971 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:09:48.972 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T18:09:48.972 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-20T18:09:48.973 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-20T18:09:48.983 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T18:09:48.983 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:09:48.983 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:09:48.983 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.85808 2026-02-20T18:09:48.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T18:09:48.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T18:09:48.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2401: main: code=0 2026-02-20T18:09:48.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: teardown td/test-erasure-eio 0 2026-02-20T18:09:48.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-20T18:09:48.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-20T18:09:48.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-20T18:09:48.985 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-20T18:09:48.985 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-20T18:09:48.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-20T18:09:48.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-20T18:09:48.985 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-20T18:09:48.987 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-20T18:09:48.987 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-20T18:09:48.988 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-20T18:09:48.988 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-20T18:09:48.989 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-20T18:09:48.989 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-20T18:09:48.989 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-20T18:09:48.990 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:09:48.990 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-20T18:09:48.991 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-20T18:09:48.991 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T18:09:48.991 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-20T18:09:48.992 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-20T18:09:48.993 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-20T18:09:48.994 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-20T18:09:48.994 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-20T18:09:48.994 INFO:tasks.workunit.client.0.trial195.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85808 2026-02-20T18:09:48.994 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.85808 2026-02-20T18:09:48.995 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-20T18:09:48.995 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-20T18:09:48.995 INFO:tasks.workunit.client.0.trial195.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2406: main: return 0 2026-02-20T18:09:48.996 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-20T18:09:48.996 DEBUG:teuthology.orchestra.run.trial195:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-20T18:09:49.061 INFO:tasks.workunit:Stopping ['erasure-code'] on client.0... 2026-02-20T18:09:49.061 DEBUG:teuthology.orchestra.run.trial195:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-20T18:09:49.418 DEBUG:teuthology.parallel:result is None 2026-02-20T18:09:49.418 DEBUG:teuthology.orchestra.run.trial195:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2026-02-20T18:09:49.443 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2026-02-20T18:09:49.443 DEBUG:teuthology.orchestra.run.trial195:> rmdir -- /home/ubuntu/cephtest/mnt.0 2026-02-20T18:09:49.500 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2026-02-20T18:09:49.500 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-20T18:09:49.517 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-20T18:09:49.518 DEBUG:teuthology.orchestra.run.trial195:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-20T18:09:49.669 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-20T18:09:49.681 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-20T18:09:49.681 DEBUG:teuthology.orchestra.run.trial195:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-20T18:09:49.698 INFO:teuthology.orchestra.run.trial195.stderr:bash: line 1: ntpq: command not found 2026-02-20T18:09:49.727 INFO:teuthology.orchestra.run.trial195.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-20T18:09:49.727 INFO:teuthology.orchestra.run.trial195.stdout:=============================================================================== 2026-02-20T18:09:49.727 INFO:teuthology.orchestra.run.trial195.stdout:^+ B1-66ER.matrix.gs 2 6 277 110 +2874us[+2882us] +/- 41ms 2026-02-20T18:09:49.727 INFO:teuthology.orchestra.run.trial195.stdout:^+ 157.245.125.229 2 6 377 47 +22us[ +28us] +/- 42ms 2026-02-20T18:09:49.727 INFO:teuthology.orchestra.run.trial195.stdout:^+ lithium.constant.com 2 6 377 113 +30us[ +74us] +/- 53ms 2026-02-20T18:09:49.727 INFO:teuthology.orchestra.run.trial195.stdout:^* ntp2.zeus.frumentum.media 2 6 377 45 -2001us[-1995us] +/- 25ms 2026-02-20T18:09:49.728 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-20T18:09:49.740 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-20T18:09:49.740 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-20T18:09:49.754 DEBUG:teuthology.orchestra.run.trial195:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-20T18:09:49.823 DEBUG:teuthology.orchestra.run.trial195:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-20T18:09:49.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-20T18:09:49.848 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-20T18:09:49.860 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-20T18:09:49.872 INFO:teuthology.task.internal:Duration was 994.600123 seconds 2026-02-20T18:09:49.872 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-20T18:09:49.887 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-20T18:09:49.887 DEBUG:teuthology.orchestra.run.trial195:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-20T18:09:49.927 INFO:teuthology.orchestra.run.trial195.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-20T18:09:50.260 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-20T18:09:50.260 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial195.front.sepia.ceph.com 2026-02-20T18:09:50.260 DEBUG:teuthology.orchestra.run.trial195:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-20T18:09:50.318 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-20T18:09:50.319 DEBUG:teuthology.orchestra.run.trial195:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-20T18:09:50.419 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-20T18:09:50.419 DEBUG:teuthology.orchestra.run.trial195:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-20T18:09:50.442 INFO:teuthology.orchestra.run.trial195.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-20T18:09:50.442 INFO:teuthology.orchestra.run.trial195.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-20T18:09:50.443 INFO:teuthology.orchestra.run.trial195.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-20T18:09:50.443 INFO:teuthology.orchestra.run.trial195.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-20T18:09:50.444 INFO:teuthology.orchestra.run.trial195.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-20T18:09:50.452 INFO:teuthology.orchestra.run.trial195.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 81.4% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-20T18:09:50.454 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-20T18:09:50.474 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-20T18:09:50.474 DEBUG:teuthology.orchestra.run.trial195:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-20T18:09:50.521 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-20T18:09:50.537 DEBUG:teuthology.orchestra.run.trial195:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-20T18:09:50.585 INFO:teuthology.orchestra.run.trial195.stdout:kernel.core_pattern = core 2026-02-20T18:09:50.616 DEBUG:teuthology.orchestra.run.trial195:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-20T18:09:50.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-20T18:09:50.673 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-20T18:09:50.689 INFO:teuthology.task.internal:Transferring archived files... 2026-02-20T18:09:50.692 DEBUG:teuthology.misc:Transferring archived files from trial195:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/yuriw-2026-02-20_16:06:38-rados-wip-pdonnell-testing-20260219.182737-tentacle-distro-default-trial/60966/remote/trial195 2026-02-20T18:09:50.694 DEBUG:teuthology.orchestra.run.trial195:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-20T18:09:50.749 INFO:teuthology.task.internal:Removing archive directory... 2026-02-20T18:09:50.749 DEBUG:teuthology.orchestra.run.trial195:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-20T18:09:50.798 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-20T18:09:50.811 INFO:teuthology.task.internal:Not uploading archives. 2026-02-20T18:09:50.811 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-20T18:09:50.827 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-20T18:09:50.827 DEBUG:teuthology.orchestra.run.trial195:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-20T18:09:50.854 INFO:teuthology.orchestra.run.trial195.stdout: 917572 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 20 18:09 /home/ubuntu/cephtest 2026-02-20T18:09:50.855 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-20T18:09:50.866 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-20T18:09:50.900 INFO:teuthology.run:Summary data: description: rados/standalone/{supported-random-distro$/{centos_latest} workloads/erasure-code} duration: 994.6001229286194 flavor: default owner: scheduled_yuriw@soko04.front.sepia.ceph.com success: true 2026-02-20T18:09:50.900 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-20T18:09:51.005 INFO:teuthology.run:pass